1. #1
    Ganchrow
    Nolite te bastardes carborundorum.
    Ganchrow's Avatar Become A Pro!
    Join Date: 08-28-05
    Posts: 5,013
    Betpoints: 980

    Pinnacle XML line feed

    I've recently starting using Pinnacle's XML line feed again (to Pinny's specifications) but am extremely disappointed with the speed of service -- especially during peak times when it becomes essentially unusable.

    Is anyone having any luck with this? I think I may need to go back to scraping their HTML.

    SBR Founder Join Date: 8/28/2005


  2. #2
    jjgold
    Update your status
    jjgold's Avatar MLB Moderator
    Join Date: 07-20-05
    Posts: 296,537
    Betpoints: 56

    Scrape their html and your ip wll be blocked if you put some auto reloader in it.

    XML is ok but again your IP will be blocked if you grab the feed more than every 2 minutes or so.

    I love you Ganch but your not playing smart today at all.

    BTP
    Week 6
    4-1-0 412 pts

    BTP
    Week 5
    4-1-0 446 pts

    SBR
    Sharp
    Bettor

    $$$$$

    Bald
    & Proud
    of Myself -jjgold

    SBR Founder Join Date: 7/20/2005


  3. #3
    ShamsWoof10
    ShamsWoof10's Avatar Become A Pro!
    Join Date: 11-15-06
    Posts: 4,827
    Betpoints: 24

    what is the differance between their XML feed and that "Dynamic Line Dsiplay" ? I use DLD alot and it's pretty sweet.. what is XML like compared to this..?

    THANKS

  4. #4
    Ganchrow
    Nolite te bastardes carborundorum.
    Ganchrow's Avatar Become A Pro!
    Join Date: 08-28-05
    Posts: 5,013
    Betpoints: 980

    Quote Originally Posted by jjgold View Post
    Scrape their html and your ip wll be blocked if you put some auto reloader in it.
    If you're going to scrape Pinnacle's HTML you just need to be careful ...

    Quote Originally Posted by jjgold View Post
    XML is ok but again your IP will be blocked if you grab the feed more than every 2 minutes or so.
    They ask for 60 seconds between requests. That's pretty easy to do when each request takes 5 minutes to fulfill.

    Quote Originally Posted by jjgold View Post
    I love you Ganch but your not playing smart today at all.
    You can tell me about it when we go out. At this point we'd even meet you in Newark.

    SBR Founder Join Date: 8/28/2005


  5. #5
    Ganchrow
    Nolite te bastardes carborundorum.
    Ganchrow's Avatar Become A Pro!
    Join Date: 08-28-05
    Posts: 5,013
    Betpoints: 980

    Quote Originally Posted by ShamsWoof10 View Post
    what is the differance between their XML feed and that "Dynamic Line Dsiplay" ? I use DLD alot and it's pretty sweet.. what is XML like compared to this..?
    You'd use the XML line feed if you wanted raw data for analytical purposes. Attached is a sample, pared-down Pinnacle XML line file (change the extension from .txt to .xml in order to view in-browser).
    Attached Files

    SBR Founder Join Date: 8/28/2005


  6. #6
    ShamsWoof10
    ShamsWoof10's Avatar Become A Pro!
    Join Date: 11-15-06
    Posts: 4,827
    Betpoints: 24

    my browser couldn't get it to come up... can you cut and paste on your post a sample..?

    Thanks

  7. #7
    Ganchrow
    Nolite te bastardes carborundorum.
    Ganchrow's Avatar Become A Pro!
    Join Date: 08-28-05
    Posts: 5,013
    Betpoints: 980

    Quote Originally Posted by ShamsWoof10 View Post
    my browser couldn't get it to come up... can you cut and paste on your post a sample..?

    Thanks
    You can also open it up with notepad (easier if you put back the .txt extension).

    SBR Founder Join Date: 8/28/2005


  8. #8
    Value
    Value's Avatar Become A Pro!
    Join Date: 02-19-07
    Posts: 9

    Quote Originally Posted by Ganchrow View Post
    I think I may need to go back to scraping their HTML.

    what is scraping html?

  9. #9
    Ganchrow
    Nolite te bastardes carborundorum.
    Ganchrow's Avatar Become A Pro!
    Join Date: 08-28-05
    Posts: 5,013
    Betpoints: 980

    Quote Originally Posted by Value View Post
    what is scraping html?
    See http://en.wikipedia.org/wiki/HTML_scraping.

    SBR Founder Join Date: 8/28/2005


  10. #10
    ShamsWoof10
    ShamsWoof10's Avatar Become A Pro!
    Join Date: 11-15-06
    Posts: 4,827
    Betpoints: 24

    What's the advantage to scraping and what's the advantage to xml feeds..? It looks like a buch of bionary code... Forgive me for being unaware but I can't make heads or tails of it... Just looks like bionary code..

  11. #11
    slash
    slash's Avatar Become A Pro!
    Join Date: 08-10-05
    Posts: 1,000
    Betpoints: 1188

    If you use spider programs, the xml feed is nice.

    SBR Founder Join Date: 8/10/2005


  12. #12
    ShamsWoof10
    ShamsWoof10's Avatar Become A Pro!
    Join Date: 11-15-06
    Posts: 4,827
    Betpoints: 24

    Spider programs..? oh sh*t.. what the hell are those..? sounds interesting...

  13. #13
    Ganchrow
    Nolite te bastardes carborundorum.
    Ganchrow's Avatar Become A Pro!
    Join Date: 08-28-05
    Posts: 5,013
    Betpoints: 980

    Quote Originally Posted by ShamsWoof10 View Post
    What's the advantage to scraping and what's the advantage to xml feeds..? It looks like a buch of bionary code... Forgive me for being unaware but I can't make heads or tails of it... Just looks like bionary code..
    Well first off, binary code will look something this:
    00000000 00010001 00000100 11111111 00010011 00000111 00000100 00010010 00000100 11111111 00001111 00000100 00001110 00001111 00001011 00000100 11111111 00010001 00000100 00000000 00001011 00001011 00011000 11111111 00010011 00000111 00000000 00010011 11111111 00000101 00010100 00000010 00001010 00001000 00001101 00000110 11111111 00010010 00010011 00010100 00001111 00001000 00000011 11111111 11111111
    While XML may not always be transparently human readable it's still much higher level and easier to visually parse than binary (not that a properly motivated person couldn't read binary with a modest effort ...)

    The advantage to an XML feed over scraping is that it's provided by design and follows a guaranteed format. HTML pages may will frequently have their formatting modified and will so without warning or description . Furthermore, many companies look askance and those scraping their pages. XML should be the default choice.

    As a general rule, there really is no advantage to scraping over XML. However, there may well be case-specific reasons why XML could be preferable.

    For example:
    • Pinnacle's XML feed does not provide access to the lines reflected in its drop-down box prices. If you wanted to automatically read this data you couldn't do so using their standard XML and instead would need to scrape the HTML.
    • A sportsbook may not devote the same quality of technology to its XML provision server as it might to its HTML provision server (or servers). The reason for this is obvious. It's more profitable to invest resources to a website used by nearly all of its customers, than to a feed used by a relatively small number. One effect of this could be what I mentioned in the first post of this thread; namely that accessing the feed could become extremely slow process. Another effect could be an unreliable XML feed that often crashed.
    • A company can effectively exert a lot more control over its XML feed than over its HTML.. The reason is b/c if a company sees you accessing its feed, it knows exactly what you're doing and can hence forbid you from accessing it too frequently. If you're scanning a web page, however, a book can't know for certain what you're doing and will typically allow you more latitude in terms of query frequency.
    • Data providers that provide HTML data, sometimes charge money for dedicated XML feeds.

    Anyway, this is by no means a complete list, but is just a few thoughts off the top of my head. These reasons (which may not even exist in any particular circumstance). may or may not outweigh the pronounced structural advantages of XML. Use XML unless you have a good reason not to.

    SBR Founder Join Date: 8/28/2005


  14. #14
    betplom
    maniac
    betplom's Avatar Become A Pro!
    Join Date: 09-20-06
    Posts: 13,444

    I can't get it to work either.

  15. #15
    tacomax
    SBR Problem Poster 2007-08
    tacomax's Avatar Become A Pro!
    Join Date: 08-10-05
    Posts: 9,621

    betplom nicky bumps!!!!!11111111111

    SBR Founder Join Date: 8/10/2005


  16. #16
    terrytateolb
    terrytateolb's Avatar Become A Pro!
    Join Date: 05-25-09
    Posts: 1

    Pinnacle HTML Scraping

    Quote Originally Posted by Ganchrow View Post
    If you're going to scrape Pinnacle's HTML you just need to be careful ...
    I've been writing a program that gets odds from Pinnacle by scraping their HTML, but earlier today my IP was banned by them. When you refer to being careful, what do you mean, i.e. what could I do differently that would prevent me from getting banned in the future?

  17. #17
    jjgold
    Update your status
    jjgold's Avatar MLB Moderator
    Join Date: 07-20-05
    Posts: 296,537
    Betpoints: 56

    Terry you can only grab every 2 minutes if not they ban your IP

    BTP
    Week 6
    4-1-0 412 pts

    BTP
    Week 5
    4-1-0 446 pts

    SBR
    Sharp
    Bettor

    $$$$$

    Bald
    & Proud
    of Myself -jjgold

    SBR Founder Join Date: 7/20/2005


  18. #18
    fortunoman
    fortunoman's Avatar Become A Pro!
    Join Date: 11-02-08
    Posts: 2

    Quote Originally Posted by Ganchrow View Post
    I've recently starting using Pinnacle's XML line feed again (to Pinny's specifications) but am extremely disappointed with the speed of service -- especially during peak times when it becomes essentially unusable.

    Is anyone having any luck with this? I think I may need to go back to scraping their HTML.
    Ganch-

    tried to PM you to see if you wanted to exchange any feeds, but apparently your Inbox is full and the system wont deliver anymore PMs to you until you clear some space.

    i have ~20 xml feeds for various books. i'd be willing to do a 1:1 exchange for feeds i dont have if there is any on my list that interests anybody reading this. anybody pls feel free to PM me if you are interested:

    Feeds I have:
    CentreBet
    Gamebookers
    Island Sports
    NordicBet
    Partybets
    Sports Interaction
    Stan James
    WSEX
    Bookmaker
    Sportbet
    Pinnacle
    Linesmaker
    Legends
    Gamblers Palace
    Betfred
    BetJamaica
    BlueSquare
    BetClick
    William Hill
    BetAtHome
    BetGun

  19. #19
    chantech
    chantech's Avatar Become A Pro!
    Join Date: 10-19-09
    Posts: 27

    Does anyone know what Pinny's limitations on screen scraping their site are? I.E. If I do it once every 3 minutes I should be ok?

  20. #20
    MonkeyF0cker
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,126
    Betpoints: 973

    Why would you scrape when they have the XML feed?

  21. #21
    jjgold
    Update your status
    jjgold's Avatar MLB Moderator
    Join Date: 07-20-05
    Posts: 296,537
    Betpoints: 56

    Approx 2-3 minutes

    Why scrape if you can just use sbrodds or another line service?

    BTP
    Week 6
    4-1-0 412 pts

    BTP
    Week 5
    4-1-0 446 pts

    SBR
    Sharp
    Bettor

    $$$$$

    Bald
    & Proud
    of Myself -jjgold

    SBR Founder Join Date: 7/20/2005


  22. #22
    MonkeyF0cker
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,126
    Betpoints: 973

    Umm. Some people want to populate line history databases in real time, jj.

  23. #23
    chantech
    chantech's Avatar Become A Pro!
    Join Date: 10-19-09
    Posts: 27

    Quote Originally Posted by jjgold View Post
    Approx 2-3 minutes

    Why scrape if you can just use sbrodds or another line service?
    Correct me if I'm wrong, but I'm assuming sbrodds doesn't give both odds and line changes in real time?

    We want to do our own analysis of how lines are changing from the time the line is posted to figure out the most optimal time to make offers and accept offers on a betting exchange and continuously optimize the prices we're getting.

  24. #24
    chantech
    chantech's Avatar Become A Pro!
    Join Date: 10-19-09
    Posts: 27

    Quote Originally Posted by MonkeyF0cker View Post
    Why would you scrape when they have the XML feed?
    I figured it would be faster since I didn't need the data for all sports and I didn't want to read in the whole XML feed. It's also easier to screen scrape than parse XML based on some of the tools we're using (I know that normally it's easier to parse XML).

  25. #25
    MonkeyF0cker
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,126
    Betpoints: 973

    Even if parsing the XML is less efficient with whatever language you are using (although I'm unaware of one where this is true), the process will still be more efficient as you'll likely need multiple scrapes for even the same sport (Full game, 1H, 2H, etc.). The full XML file is only processed once anyway. From there, you input a timestamp in your feed request and you only get line changes.

    If you're set on scraping, however, I would simply call them and ask. Otherwise, you run the risk of having your IP blocked.

  26. #26
    jjgold
    Update your status
    jjgold's Avatar MLB Moderator
    Join Date: 07-20-05
    Posts: 296,537
    Betpoints: 56

    your IP will get banned scrapping lines from Pinny

    Even if you use xml feed you only better pull it every 2-3 minutes or your done.

    BTP
    Week 6
    4-1-0 412 pts

    BTP
    Week 5
    4-1-0 446 pts

    SBR
    Sharp
    Bettor

    $$$$$

    Bald
    & Proud
    of Myself -jjgold

    SBR Founder Join Date: 7/20/2005


  27. #27
    MrX
    MrX's Avatar Become A Pro!
    Join Date: 01-10-06
    Posts: 1,540

    Their own instructions state that once per minute is acceptable.

  28. #28
    MonkeyF0cker
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,126
    Betpoints: 973

    JJ, stick to videos please. As MrX said, the XML feed can be pulled every 60 seconds without recourse.

  29. #29
    dirk93
    dirk93's Avatar Become A Pro!
    Join Date: 10-17-09
    Posts: 47
    Betpoints: 72

    Sorry to bring up and old topic but I was messing around with the XML feed and I think I set it to scrape too frequently and got blocked from Pinnacle. Anyone have any ideas on how to get unblocked? Also, I'm having trouble with the XML feed. I know the instructions are there I just can't seem to get it into a functional form. Any help?

  30. #30
    durito
    escarabajo negro
    durito's Avatar Become A Pro!
    Join Date: 07-03-06
    Posts: 13,182

    email them and ask them to unblock your ip. tell them you won't hit it more than once a minute again.

  31. #31
    squallsquall
    squallsquall's Avatar Become A Pro!
    Join Date: 05-24-09
    Posts: 273

    Anyone used the "sportType" parameter when calling the XML-feed? I'm only interested in MMA, but I can't very well set the parameter value to "Mixed Martial Arts" (which is the sport type name), as it includes spaces.

    Any suggestions? Customer service won't help out with this.

  32. #32
    Aperjan
    Aperjan's Avatar Become A Pro!
    Join Date: 09-23-10
    Posts: 1

    Hi Everyone:
    who can understand and parse pinny xml?
    i can't understand what means lastGame and lastContest in xml document? what is it? for what is it? what is last?
    how can i get only soccer data ? hopw can i get only one game data by gamenumber? is it possible?
    what is contestantnum in participand node? what is rotnum in participant node?
    what is period_status and period_updae? where i can read full manual for this xml file?

  33. #33
    creeky
    creeky's Avatar Become A Pro!
    Join Date: 09-08-08
    Posts: 2

    Quote Originally Posted by Aperjan View Post
    Hi Everyone:
    who can understand and parse pinny xml?
    i can't understand what means lastGame and lastContest in xml document? what is it? for what is it? what is last?
    how can i get only soccer data ? hopw can i get only one game data by gamenumber? is it possible?
    what is contestantnum in participand node? what is rotnum in participant node?
    what is period_status and period_updae? where i can read full manual for this xml file?
    Any help? I'm also interested in the meanings for all of the elements

  34. #34
    katstale
    katstale's Avatar SBR PRO
    Join Date: 02-07-07
    Posts: 3,791
    Betpoints: 36

    Pinny has indicated they might put buy/sell lines in the xml feed in the future. This might be worth a few folks sending them emails asking when this updated feature will be available.

    BTP
    Week 3
    4-1-0 299 pts


  35. #35
    whitey
    SBROdds Developer
    whitey's Avatar Become A Pro!
    Join Date: 04-01-08
    Posts: 490
    Betpoints: 6343

    Quote Originally Posted by creeky View Post
    Any help? I'm also interested in the meanings for all of the elements
    Pinnacle's feed is what we call "incremental" which means that when you request it you have the option of only receiving events/lines that have updated since your last call.

    I suggest sticking with the parameter last parameter as its a little simpler to use. Basically when you receive the feed for the first time they give you a timestamp \\pinnacle_line_feed\PinnacleFeedTime you save this and send it as the value for the last parameter next time you request the feed. Doing so will cause only the lines and events that have changed since the last time you requested the feed to show up.

    To request by only one league use the sportType parameter. For example, to request only soccer

    http://xml.pinnaclesports.com/pinnac...ortType=Soccer

12 Last
Top