Scraping SBObet, worth a shot?

**HAPPY BOY** · 12-13-13, 05:24 AM

man this place has changed

**smitch124** · 12-13-13, 08:17 AM

Happy I think this calls for a cartoon that depicts this change at SBR.

**03fan** · 12-13-13, 10:34 AM

What does a scraper do for you? Help get lines before movements?

**Kelly_F** · 12-13-13, 03:34 PM

Originally posted by 03fan

What does a scraper do for you? Help get lines before movements?

Erm, no, because then it would be able to predict the future. So you would be better off getting next weeks lotto results than lines off SBO.

If you mean does a scraper alert you to line movements, it is very challenging to build a scraper fast enough to do that. You would have to constantly collect data to do so an getting blocked is one of the issues you would face.

**Kelly_F** · 12-13-13, 03:35 PM

Originally posted by slobib

I need it to compare with local bookies and get the best odds fastest.
There are sites that do this but i think they lack quality and arent optimal.

In reference to local bookies, what country are you in?

**Kelly_F** · 12-13-13, 03:40 PM

Originally posted by HUY

You won't have any problems with the amount of data. You will be blocked if you request pages too often. Just download every few minutes and you should be fine.

I would hazard a guess that he wants the data faster/more regularly than that, which is of course the challenge with writing a scraper.

**Maverick22** · 12-13-13, 03:43 PM

How is setting a timeout/sleep time a challenge?

**Kelly_F** · 12-13-13, 04:05 PM

Originally posted by Maverick22

How is setting a timeout/sleep time a challenge?

I contract a couple of programmers to write/maintain applications for me, and so I don't specifically understand what you mean by that.

However SBO's odds are very dynamic, very fast. To meaningfully catch those movements a scraper would need to be collecting data as regularly as possible. I'm talking every few seconds or less. That's why the XML feed would be preferable to the OP, but he doesn't have access.

I don't know what a timeout/sleep time is but if it enables you to scrape a site every few seconds please explain.

**Maverick22** · 12-13-13, 04:45 PM

while(1) **

doWorkToScrapeData();

sleep(2000);

**

protected final void sleep() **

if (SLEEP_TIME == 0) **
return;
**

try **

logger.debug("Application Sleeping For **** seconds", (SLEEP_TIME / 1000));
Thread.sleep(getSleepTimeMinusWarning()) ;

logger.debug("Application Resuming In **** seconds", (SLEEP_TIME_WARNING / 1000));
Thread.sleep(SLEEP_TIME_WARNING);
** catch (InterruptedException ex) **
logger.error("Exception Occurred While Trying To Sleep");
logger.error(ex.getMessage());
**
**

**Maverick22** · 12-13-13, 04:46 PM

SBR doesnt seem like curly braces. but that is what i mean. the actual scraping code is not set on a time out. it just executes some tasks. and some code that controls the scraper will designate when to sleep.

simplifies the scraper itself makes it easier to manage how the sleeping is controlled.

**Kelly_F** · 12-13-13, 05:10 PM

Sorry, I missed a few words out of my post. I meant to say I am not a programmer myself (but contract a couple to write applications for me).

So whilst I have experience and understanding of some of the challenges that they face, I don't understand your code.

What I am asking is why do you think that setting a timeout/sleep time will enable data to be collected frequently, hitting the site many times in a minute.

**Maverick22** · 12-13-13, 05:47 PM

I don't really understand the question.

By creating functionality to sleep. you can repeatedly call the same code over and over again and simply configure how much time between each call.

So you dictact: "go to the pinnacle odds xml page, download and process the xml and then insert it into my database. Then... sleep (or wait) for 3000 milliseconds and then do it all over again, until i tell you to stop."

**Kelly_F** · 12-13-13, 05:57 PM

configuring the time between each call meaning how often the scraper fetches/collects the data right?

That's fine, if he is using the Pinnacle API, but it' different if he is scraping a site. Scrape it too often, and he will get blocked. He has to put things in place to prevent that.

And scraping a site is not as fast as using an XML feed.

The point I am making is that I think I an tell from the OP's posts that speed is important to him. When the odds change, he wants to know about it instantly, or at least as fast as possible.

**BigDaddy777** · 02-12-14, 03:05 AM

<br>I've written scrapers for many websites and I can scrape SBObet for you, contact me
Do you have any pre-written for this site - > http://scoresandodds.com?

**BigDaddy777** · 02-13-14, 09:18 PM

Data Scraping from Websites

Here's some feedback I received from a programmer when I sent him the comments about scrapers from this forum -

=====

You mentioned someone said "I use python and LXML, and some tool for firefox "page inspector" i can get any info from a page within a few minutes."

Well, there are different scraping techniques. There is also something called cUrl technology. The site to be crawled determines which is the most suitable technology for the job.

You also mentioned someone wrote "Scoresandodds.com offer an XML feed API for their match ups and odds covering all sports. I consulted with a group who used it..."

I don't think this is a true statement. The site doesn't have any api listing. Again the statement is contradictory. Either they provide XML feed or API. *XML feed API* is a wrong phrase.

=====

Any reaction to this?

**BigDaddy777** · 02-17-14, 01:42 AM

<br>Don't all chime in at once...

baha

**flyingillini** · 02-26-14, 11:45 AM

One of the best books in the business, been using them for years.

**SportMac** · 02-26-14, 12:01 PM

also there are lots of application those can track your account and if found robotic, it will be freezed.

**SportMac** · 02-26-14, 12:03 PM

Originally posted by Kelly_F

configuring the time between each call meaning how often the scraper fetches/collects the data right?

That's fine, if he is using the Pinnacle API, but it' different if he is scraping a site. Scrape it too often, and he will get blocked. He has to put things in place to prevent that.

And scraping a site is not as fast as using an XML feed.

The point I am making is that I think I an tell from the OP's posts that speed is important to him. When the odds change, he wants to know about it instantly, or at least as fast as possible.

Do someone have any sample of XML feed. I also wanna to make a progreame that can help in XML feed.

**lamichaeljames** · 07-08-14, 04:54 PM

Originally posted by SportMac

also there are lots of application those can track your account and if found robotic, it will be freezed.

agreed but there are ways around it.

**nmlfree** · 07-09-14, 04:16 AM

Arwar, i don't have enough posts to send you a PM but i'm interested in your scraper, can you send a PM with your email or skype so we can discuss further details?

Thanks,

Eric

**bihon** · 07-10-14, 04:51 AM

Originally posted by pringles

well, im using both very skilled designer and programmer, we have done the interface and the programming starts in a few days.
im paying around 3k€ for the whole set-up

The price looks fair, but if you are paying €3000 for that, you probably don't bet peanuts and you depend on reliability of data. Why risking when you can order paid feed in multiple formats from reliable companies?