python - Scrapy sqmllinkextractor rule changes urls -


I am trying to create a webcrawler spider using a scraper and I have a rule for the intended link to my crawler To create a regex using: 

  rule = (rule (SgmlLinkExtractor (allow = 'http: \ \ / \ www \ .cartelera \ .com \ .uy \ / apeliculafunciones \ .aspx \ [^ "] * 1 and 2 9 '),' Parse_loli, follow or = True)) that   
 I examined Rijeks and it works And this link I want to see the spider To do this, but when I ran the program, the spider did not get any items and with little investigation I came to know that whenever the RezX matches the item correctly, when the spider goes to crawl the page So that URL should be used as much as you want.  
 For example, spiders should match and crawl URLs such as  
  http: / /www.cartelera.com.uy/apeliculafunciones.aspx?7242&&; cine and o Ra & A 1 and 2 9 http://www.cartelera.com.uy/apeliculafunciones.aspx?10704&&CINE&OBRA&1&29 http://www.cartelera.com.uy/apeliculafunciones .aspx? 10697 & amp; Amp; & Amp; And & amp; Obara & amp; 1 & amp; Amp; 2 & amp;    
 but come out like url scrapy crawls  
  http: // www cartelera.com.uy/apeliculafunciones.aspx?-1 = & Amp; 29 = & amp; 7242 = & amp; CINE = & amp; OBRA = http://www.cartelera.com.uy/apeliculafunciones.aspx?-1=& 29 = & amp; 10704 = & amp; Cine = & amp; OBRA = http://www.cartelera.com.uy/apeliculafunciones.aspx?-1=&&29=&10697=&CINE=&OBRA=   
 these URLs Errors take pages, obviously, no items were found.  
 There is no error message why this is happening and how do I avoid it?  
  
  class MySpider (crawlspider): name = 'cartelera' permission_domain = ["www.cartelera.com.uy"] start_urls = ["http: // www2 .cartelera.com.uy / apeliculafunciones.aspx? ,, PELICULAS, OBRA, 0,26 "] = rule (rule (SgmlLinkExtractor (allow = 'http: \ \ / \ / www \ .cartelera \ .com \. U / Apelikulfunsins \. Spaks \ "[^"] * 1 and 2 9 '),' Parse_loli, follow or = True),) were simple debugging method and Czech crawled to ## output urls def parse_loly (self , Response): print '% s'% response.url returns    
 
 
 those URLs There are no standards, you can overcome this by saying not to remove SGML, like to do them canonicalize:  
  SgmlLinkExtractor (allow = 'http ...', canonicalize = False)   
 Here's how you work with Scapery Shell:  
  $ scrapy shell "http: // www2 Cartelera.com.uy/apeliculafunciones Aspx ?, PELICULAS, Obra, 0,26 "... ... gt; & Gt; & Gt; SgmlLinkExtractor from Scrapy.contrib.linkextractors.sgml Import & gt; & Gt; & Gt; S = SgmlLinkExtractor (allowed = 'http: \ / \ / www \ .cartelera \ .com \ .uy \ / apeliculafunciones \ .aspx \? [^ "] * 1 and 29', canonicalize = false)> gt; & gt For links in (;): ... Print Link ... link (url = 'http: & amp; //www.cartelera.com.uy/apeliculafunciones.aspx 10697 & amp; Cine & amp; OBRA & amp; - 1 & amp; 29 ', text = u' ', fragment =' ', nofollow = False) link (url =' http: //www.cartelera.com.uy/apeliculafunciones. Aspx? 10697 & amp; amp; amp; & amp;; & amp;; & amp;; & amp; # 29; titulosalas '; text = u'aqu \ xed', fragment = '', nofollow = False) link (url = 'Http: //www.cartelera.com.uy/apeliculafunciones.aspx? 10795 & amp; Cine and Obra and 1 & 29', text = u '', block = '', nofollow = False) ...

Comments

Post a Comment

Popular posts from this blog

c - Mpirun hangs when mpi send and recieve is put in a loop -

I was trying to use the program to use mifrewan on 4 node clusters. Distributing node 0 data in nodes 1, 2 and 3. In the program, from '90 'to variable values ​​of' dior 'to be calculated. Therefore the node is distributing data and collecting results in an omitted fashion (for different values ​​of var 'dir') when do {*******} Whereas (dir Loop is given, Mippun is hanging, and no output is found. But when I comment on two {*******} (dir The loop output is obtained for the initial value of variable dir dir = -90 ), and that output is correct. The problem occurs when the loop is given. Can anyone help me in solving this problem. #include "mpi.h" int main (int argc, char * argv []) Float dir = -90; Int rank, nanoprakash; MPI_Status status; MPI_Init (& amp; argc, & argv); MPI_Comm_rank (MPI_COMM_WORLD, & amp; Rank); MPI_Comm_size (MPI_COMM_WORLD, & amp; numprocs); If (rank == 0) {/ / data to begin / / (dest = 1; dest ...
Read more

python - Apply coupon to a customer's subscription based on non-stripe related actions on the site -

I can not find anything in the source about using coupons after subscribing to a user's plan. Is it possible to use a coupon to credit a user's membership after initial subscription? If not, what is the best option? Specifically, I want to give a user 100% coupon for that month, when they take 5 [site specific actions]. This is possible, but may not be the best idea. Coupons are currently applied to the stripe, then apply again at the customer level and each client can have a single coupon. So if you are already offering discounts to the coupon, then the customer who is currently in the present will also give up. It also means that if you are using a recently-added multiple subscription capacity, then all of the user's subscription will be ready for the month. Coupons can be set to "Once", only once (as the only name) can be used by any customer, so if this credit can earn more than once, You will have to maintain a 100% discount coupon continuousl...
Read more

java - Unable to get JDBC connection in Spring application to MySQL -

मैंने स्प्रिंग में डीबी कॉन्फ़िगरेशन का पालन किया है: & lt; bean id = "dataSource" Class = "org.apache.commons.dbcp.BasicDataSource" नष्ट-विधि = "बंद" & gt; & Lt; प्रॉपर्टी नाम = "url" मान = "jdbc: dburl" / & gt; & Lt; संपत्ति नाम = "driverClassName" मान = "com.mysql.jdbc.Driver" / & gt; & Lt; संपत्ति नाम = "उपयोगकर्ता नाम" मान = "उपयोगकर्ता नाम" / & gt; & Lt; संपत्ति नाम = "पासवर्ड" मान = "पासवॉर्फ" / & gt; & Lt; प्रॉपर्टी नाम = "हटाए गए हटाए गए" मान = "सही" / & gt; & Lt; गुण नाम = "प्रारंभिक आकार" मान = "3" / & gt; & Lt; संपत्ति नाम = "अधिकतमअक्टिव" मान = "5" / & gt; & Lt; / सेम ​​& gt; डीबी से गुण मूल्यों को लोड करने के लिए अतिरिक्त कॉन्फ़िगरेशन। & lt; bean id = "config1" class = "org.apache.commons.configuration....
Read more