[akphidelt]

Their old site was easy. Now they use some Ajax loader to prevent easy html scraping. I've tried to send headers to replicate the action but no success. If you've been able to crack it let me know. This change is killing me right now! Had a cron job that gathered spreads but it's useless now.

[daneblazer]

I'd guess that this is one reason they did the update, to prevent scrapes. They've been trying to chase away good bettors & poker players and cater to the common hamburger for the better part of their existence. Update just happened, so give it some time and I imagine someone will figure it out.

[arwar]

Pinny changed their site too, now they want to charge for lines. Benji over at Don Best says the package they offer is about $1000/month. aside from why you want Bovada data, sned me a PM and maybe i can point you in the right direction. AJAX is a pain because it can change a single value withou having to make a return trip to the server. that being said, it has sort have fallen out of favor. most of the coders for the books are now relying on jquery, probably because it's so damned overwhelming to look at the code. but i know some tricks that can get around both of those methods. i don't know if it is true but SBR used to be a pain because of nested htmls combined with a database query - so had to use the readystate method in the DOM model. But they aren't the best site to scrape anyway. I only like helping people so talk.

[akphidelt]

Pinny changed their site too, now they want to charge for lines. Benji over at Don Best says the package they offer is about $1000/month. aside from why you want Bovada data, sned me a PM and maybe i can point you in the right direction. AJAX is a pain because it can change a single value withou having to make a return trip to the server. that being said, it has sort have fallen out of favor. most of the coders for the books are now relying on jquery, probably because it's so damned overwhelming to look at the code. but i know some tricks that can get around both of those methods. i don't know if it is true but SBR used to be a pain because of nested htmls combined with a database query - so had to use the readystate method in the DOM model. But they aren't the best site to scrape anyway. I only like helping people so talk.

Thanks, I'll give it a better try this weekend. And I definitely use jQuery, but it doesn't replace AJAX, it's just a javascript library that makes AJAX easier to use by wrapping it up in easy methods. Otherwise the raw code is pretty ridiculous.

I looked at the source code for bovada, and all it's main content is created after the page loads, they have a maze of headers being sent and anonymous functions to make it very difficult. This will require some work but if they can show it on a screen it can be captured. I'm sure they provide higher end sites with an xml access. I'll keep my progress updated. Thanks again.

[EXhoosier10]

more clunky, but you could always use Selenium

[akphidelt]

more clunky, but you could always use Selenium

I really gotta check this Selenium out, looks like some fun. Thanks for the tip.

[akphidelt]

I figured it out and it was much more simple than I thought, just didn't think of it. They store all the game information in a script at the beginning, then they load the html structure without the game data with ajax, then they attach the data to the new html with a javascript function. This was tricky because the responses weren't giving me any game information, took a while to figure it out... but you actually don't need to worry about their ajax at all, just have to parse their javascript which is not that difficult. So all is good again.

[akphidelt]

Not gonna give all the code away on the Internet. If you have questions, PM me... but just to get people started in the right direction. The site doesn't allow javascript tags so when you see $cript it should be script.

$url = 'https://sports.bovada.lv/baseball/mlb/game-lines-market-group';

$html = file_get_html($url);

preg_match_all('/<$cript(.*?)<\/script>/is', $html, $matches);
    
foreach($matches[1] as $key => $match)
{
   $regex = '/type="text\/javascript">var swc_market_lists =[\s]?[^{]*([^;]*)/i';
   preg_match($regex, $match, $gameMatch);
        
   if(isset($gameMatch[1]))
   {
      $matchups = json_decode($gameMatch[1]);
   }
}

This will store the game lines in to a usable array to do whatever you want with it.

[akphidelt]

Noob here, what exactly is scraping? Compiling lines from a sportsbook for easier, more personal access?

Pretty much. You can get stats, lines, whatever you see on the internet basically can be turned in to usable data for whatever you want.