Page 1 of 1

Database update

PostPosted: Wed Jul 13, 2005 12:29 pm
by Teo
There are two sources of IP-address data
- When people are good enough to identify their city, it is stored in the DB and associated with the /24 they are currently using. This is obviously the best info. source.
- There are (currently 10) robots on the server, tracerouting to IP addresses within /24's that we haven't yet identified. There are then several hundred regular-expression rules (from sarangworld) which match parts of the names detected to countries/cities..


The Sarangworld database was built in 1999-2000.
The technic used by him is a basic IP ping and tracert system with filters with a lot of holes.
Since these last 5 years, several hundred of thousand IP have changed.
Each month, some thousand of IP change.

The update of hostip database come only from some monthly dozen manual changes by visitor lost ?
Or several server run every days to update this system ?

Re: Database update

PostPosted: Wed Jul 13, 2005 12:53 pm
by bfolkens
Teo wrote:
There are two sources of IP-address data
- When people are good enough to identify their city, it is stored in the DB and associated with the /24 they are currently using. This is obviously the best info. source.
- There are (currently 10) robots on the server, tracerouting to IP addresses within /24's that we haven't yet identified. There are then several hundred regular-expression rules (from sarangworld) which match parts of the names detected to countries/cities..


The Sarangworld database was built in 1999-2000.
The technic used by him is a basic IP ping and tracert system with filters with a lot of holes.
Since these last 5 years, several hundred of thousand IP have changed.
Each month, some thousand of IP change.

The update of hostip database come only from some monthly dozen manual changes by visitor lost ?
Or several server run every days to update this system ?


There are actually quite a few updates to the system on a regular basis.

PostPosted: Wed Jul 13, 2005 1:42 pm
by Teo
But where come from datas about cities ?
Because Sarangworld work only on US city, not on other countries.

Datas must come from another database systems because you have not the technology to extract them yourself.

But where have you find the original files ?
(provider list of Geo Ip is very short)

Can you certified than your contents don't come from some hacked files (from other providers without licences paid) ?

PostPosted: Wed Jul 13, 2005 3:47 pm
by Teo
If city links have been done by you, how do you have retrieve them ?

The Community GEO IP Project will be interesting to use the same methods to update the datas.

(note : if you have done that yourself, you are under the software patent that we speak on other topics because there are not a lot of systems to do that)

PostPosted: Wed Jul 13, 2005 4:01 pm
by Teo
I have found that on the website to explain where these files come from.

The city data comes from various places around the 'net as well as from a major data donation.


major data donation = full database from a geo provider ?
:?

It's really a GPL licence if all datas come from another private provider ?

PostPosted: Thu Jul 14, 2005 7:33 am
by bfolkens
Teo wrote:I have found that on the website to explain where these files come from.

The city data comes from various places around the 'net as well as from a major data donation.


major data donation = full database from a geo provider ?
:?

It's really a GPL licence if all datas come from another private provider ?


Major data donation = people, especially ISPs donating their block information, locale, etc. We rely on the community for updates, and as such, the community has free access to the data.

PostPosted: Thu Jul 14, 2005 7:42 am
by bfolkens
Teo wrote:But where come from datas about cities ?
Because Sarangworld work only on US city, not on other countries.

Datas must come from another database systems because you have not the technology to extract them yourself.

But where have you find the original files ?
(provider list of Geo Ip is very short)

Can you certified than your contents don't come from some hacked files (from other providers without licences paid) ?


Teo, look at the website. If you want more information, do an SVN checkout or just browse it. Everything is there for you to see.

People volunteer their information (city, etc) because they *want* to and they understand that it will help everyone in the long run. That's how these community projects work. If everyone gives a little over time, the project really begins to mature - that's how it has in this case. Time, patience, and generosity begin to pay off.

PostPosted: Thu Jul 14, 2005 11:51 am
by Teo
It mean than some millions of user have updated the database or that 1 user as put inside an database provided by other geo provider ?

Because we speak about several hundred of thousand city position.

There are not a lot of source of datas,
when you see than 1 user put inside several hundred of thousand geo location ... maybe he has not retrieve it manualy ...

If I send you a copy of my Win xp, you will put it on download server because it's a given software then free without to check the source right ?

PostPosted: Thu Jul 14, 2005 4:00 pm
by Teo
If the guy who has retrieve himself all the datas, can explain how he has done that it will be nice.

Community would like his method to implement and try it too.

:? I'm curious to know how many computers and months he has used to do that and what is his original method to avoid the US patent.

PostPosted: Thu Jul 14, 2005 6:39 pm
by bfolkens
Teo wrote:It mean than some millions of user have updated the database or that 1 user as put inside an database provided by other geo provider ?

Because we speak about several hundred of thousand city position.

There are not a lot of source of datas,
when you see than 1 user put inside several hundred of thousand geo location ... maybe he has not retrieve it manualy ...

If I send you a copy of my Win xp, you will put it on download server because it's a given software then free without to check the source right ?


I had a hard time understanding this, but I'll do my best to reply:
1) Again, lots of visitors AND lots of ISP's updating entire blocks of information at will over a long period of time.
2) WinXP is not just common and free "information" - it is copyrighted. Furthermore, users of it are granted an End-User License Agreement (EULA) that they must comply with - so you sending that data to me would violate your EULA.

Re: Database update

PostPosted: Mon Jul 21, 2008 10:54 pm
by ifsca
I may be missing something but I find no reference to this in the discussion board.

Why not have a file available for download that contains only the changes to the database over the previous 30 days?

As I understand your current policy, users of this database must download the entire database each month to get the latest updates.

Did I miss something or is there a reason this is not being done?

Amazing project by the way. Hugely ambitious. Thank you for making this available.

Re: Database update

PostPosted: Thu Jan 22, 2009 5:58 pm
by Gaspar
bfolkens wrote:There are actually quite a few updates to the system on a regular basis.


When was the last update?