python - why is mechanize not downloading the full page? -


I am using mechanisms to link and I am getting all employees of a fixed company. The results of the employees are missing out in the middle and I have no idea.

Here is my code (My Linden Sign In Notice):

  import from mechanize browser br / browser (from brs import) br.set_handle_robots (wrong) Br.open ('https://www.linkedin.com/') br.select_form ('Login') br ['session_key'] = yOUR_EMAIL_HERE Br ['session_password'] = YOUR_PASSWORD_HERE response = br.submit () page = Br.open ('https://www.linkedin.com/vsearch/p?f_CC=10667') html = page.read () soup = beautiful (html) text = soup. Vengeance () text = text ncod ("esi", "ignore") fo = open ("website.html", 'wb') fo.write (text) fo.close ()   

This is the response (I recommend downloading HTML and looking at it with a browser):

I'm not sure that I have used open work correctly; Anyway, that might be the problem, thanks in advanced! If you have any questions let me know.

OK, after doing some research it seems that the mechanisms do not wait to load JavaScript And so I was not downloading the correct information. Mechanize does not provide a method to wait JavaScript, so I have to use either windmill or selenium on these: and < / Html>

Comments

Popular posts from this blog

c - Mpirun hangs when mpi send and recieve is put in a loop -

python - Apply coupon to a customer's subscription based on non-stripe related actions on the site -

java - Unable to get JDBC connection in Spring application to MySQL -