next »

[Keebler]

This is going to be confusing.

I have a table with urls. I am trying to build a spider that will only use each url once, because when I load new urls into it, I dont want it to start all over again scraping the old urls once again.

How can I set up the table, so that when the spider queries the table for a new url, it only gets new urls?

I was thinking of setting it up something like, where along with the URL value, for unspidered urls I have a value that is automatically 0, and then once the URL has been used I switch the value to a 1. Then my query would have to be something that searches for URLS where value = 0?

my first query attempt was:

SELECT * FROM $dbUrltable LIMIT 1 WHERE create = 0

but it said that my syntax was wrong... I tried all sorts of variations but I cant figure it out.

Is my question understandable?

Basically, either how do I fix that query, or whats the best way of going about this.

Thanks in advance.

[deregular]

Thats exactly how i would do it..

Try this instead..

SELECT * FROM $dbUrltable WHERE create = 0 LIMIT 1

[Keebler]

Thanks for the reply, sorry it took so long to get back. This is what happens when I use that code that you just showed:

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'create = 0 LIMIT 1' at line 1

[nutballs]

Is create an int or string (like varchar)

[Keebler]

its an integer, default value of 0. but...

it worked when I added lil ` around the "create" column

like this

Code:

        $query = "SELECT * FROM $dbUrltable WHERE `create` = 0 LIMIT 1";

instead of

Code:

        $query = "SELECT * FROM $dbUrltable WHERE create = 0 LIMIT 1";

[nutballs]

Oh haha lol. Duh. Create is a reserved word. Forgot that. So yep the backqoutes fix that.

[Keebler]

Oh haha lol. Duh. Create is a reserved word. Forgot that. So yep the backqoutes fix that.

OH thats why? That makes sense. hahaha.

I was wondering why I had never needed backquotes before.... I guess I should just rename the column name.

[deregular]

Dammit, should have picked that up myself... 

[nutballs]

its probably not something i would normally recognize because my column naming convention would never end up with a reserved word as the column. Probably similar for you.