python - why is mechanize not downloading the full page? -
I am using mechanisms to link and I am getting all employees of a fixed company. The results of the employees are missing out in the middle and I have no idea.
Here is my code (My Linden Sign In Notice):
import from mechanize browser br / browser (from brs import) br.set_handle_robots (wrong) Br.open ('https://www.linkedin.com/') br.select_form ('Login') br ['session_key'] = yOUR_EMAIL_HERE Br ['session_password'] = YOUR_PASSWORD_HERE response = br.submit () page = Br.open ('https://www.linkedin.com/vsearch/p?f_CC=10667') html = page.read () soup = beautiful (html) text = soup. Vengeance () text = text ncod ("esi", "ignore") fo = open ("website.html", 'wb') fo.write (text) fo.close () This is the response (I recommend downloading HTML and looking at it with a browser):
I'm not sure that I have used open work correctly; Anyway, that might be the problem, thanks in advanced! If you have any questions let me know.
OK, after doing some research it seems that the mechanisms do not wait to load JavaScript And so I was not downloading the correct information. Mechanize does not provide a method to wait JavaScript, so I have to use either windmill or selenium on these: and < / Html>
Comments
Post a Comment