(PSO) particle swarm optimization , (MOEA) ,(ANN) Artificial Neural networks

**Karomesis** · 09-24-09, 10:27 PM

(MOEA) Multi-Objective Evolutionary Algorithms, (PSO) Particle Swarm Optimizer, (ANN)

Hey Guys,

a noob here obviously, but perhaps one type that you may not be accustomed to. I ask SERIOUS questions.

Not the same BS you've seen over the years from those who've come and gone.

The question is in the thread title. Does anyone here have experience with either (MOEA) Multi-Objective Evolutionary Algorithms, (PSO) Particle Swarm Optimization, or (ANN) Artificial Neural Networks? prefferably a combination of at least two of them in a "hybrid" simulation.

Just curious because linear statistics don't seem to work at all for predicting biological systems performance. ...as humans are biological systems and not "odds" and convenient equations. I've already UTFSE (used the phuking search engine) and found nothing relating to this topic so I figured I'd ask some pros...or at least those professing to be "pros".

Here is one of the papers that forms the basis of the thread
http://www2.computer.org/portal/web/...09/HIS.2006.33

here's another
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=16970&arnumber= 785467&count=100&index=2

and one more for the road..

http://www.mitpressjournals.org/doi/abs/10.1162/106365603766646816?prevSearch=authorsfie ld%253A%2528Toffolo%252C%2BAndrea%2529&s earchHistoryKey=

you'll also notice something else about me, when I say something is true or shows promise in research I REFERENCE MY SOURCES.. As should everyone else. There is a reason why it's called a "peer reveiwed study/journal". Maybe for the same reason the "double blind placebo controlled" study is the only one they use for approving multi billion dollar drugs

the last link is from MIT...you know..the guys that ripped Vegas off millions not too long ago?

not these scam artist "cappers" who post false/misleading records and rely on clever marketing tactics to lure in a gullible populace. Basic Evolutionary psychology/game theory states that they would almost certainly NEVER give away their golden goose, so please don't waste your time attempting to convince me otherwise.

After a few weeks of reading here I can tell there are a few highly intelligent people who make actual sense. And many, many more who are nothing but addicts, egotists, or just good ol fashioned degenerates.

I post on another forum, ferrarichat.com where some REALLY rich people who sometimes have 6 Ferraris post. So I know It's not impossible that there are some serious people here. Not the "so I can go on a cruise" people I mean the people making$50k-100K+ monthly. The only people worth talking with for any serious length of time.

please forgive my long winded post, I just had alot to say and would love to see if anyone out there is thinking along the same lines. I initially hesitated for fear of killing the potential "golden goose" but figured the risk would be worth it with real players. That's why I posted it in the think tank

i.e maybe we could do some grid computing and pool our computational resources together and get better deals.

thanks for any responses in advance.

**Justin7** · 09-24-09, 11:22 PM

Karo,

Why don't you explain in your own words what this stream of logic is trying to do? I skimmed one of the links, and could not make sense of it. If you want sharp minds to help you, identify the problem and proposed solution. I cannot do that quickly yet with what I read.

**MrX** · 09-24-09, 11:45 PM

Most pretentious introduction ever?

**MonkeyF0cker** · 09-25-09, 06:22 AM

Three words for the OP: Monte Carlo simulations.

**Wrecktangle** · 09-25-09, 07:56 AM

Quickie observations:

I've used Neural, doesn't work well or at least I could not make it work as well as the methods I was into at the time (regression). Evolutionary algorithms seem to be much slower than some of the Control Theory stuff I'm using now, but then again who knows if I was using that right.

Particle swarm: fun name, but I've no clue...good luck and good night.

**wintermute** · 09-25-09, 08:15 AM

Your references are just abstracts of the papers and don't really say anything. You have to pay to read the actual papers. No thanks.

Maybe my local university has some of them - I'll check.

It's been my experience that stuff like this usually has no real world applicability. I prefer to spend my time on technologies like MCMC (Markov Chain Monte Carlo) that have been used to make predictions about sports-related activities with some success.

**Karomesis** · 09-25-09, 02:11 PM

Thanks for responding guys.

Justin,

Essentially, I'm trying to use the latest research into these areas to yield a program with limited system bias.(i.e taking out as much of the human element as possible) Ideally, I'd like to get the "noise" reduced in the results from a database of sports statistics and have a program that "learns" as it goes along. MOEA is the primary method I think will help to do this, while the PSO is the way I can "teach" the ANN as it goes along. I was going to use back propogation method to teach the system, but recent breakthroughs in PSO have indicated this is not the ideal solution.

The problem is too much emphasis placed on random noise and how to reduce that as much as possible so the program can do what it does best, learn and predict.

Because we're dealing with biological systems, maybe we could use "successive generations" of the algorithm every time there is a change in the system, new player is added, new coach, varied constraints such as turf vs grass etc

There are a few white papers on using one of these to predict sporting outcomes, but I'm not too sure if what they say has any validity for real world applications. They only predicted accurately a few weeks in the NFL better than the human counterparts, not enough time and an obvious lack of sampling.

Here's the whole paper in PDF

College of Engineering | University of Wisconsin-Madison

http://homepages.cae.wisc.edu/~ece539/project/f03/kahn.pdf

The College of Engineering at the University of Wisconsin-Madison is known for outstanding research, educators and service to society.

Wrecktangle,

did you use back propogation methods for your ANN? PSO shows much more promise in teaching the ANN from what I understand.

MonkeyF0cker,

From what I've read on MCMC methods they don't take into account the learning the system needs to undergo in order to become more efficient. Because we're dealing with certain non linear patterns and alot of system noise. Do you have experience using MCMC methods? I'm always learning and would be interested in seeing how your program performs.

and here's a short summary of why I think PSO may be suited for the prediction of sporting events

"Particle swarm optimization (PSO) is a population based stochastic optimization technique developed by Dr. Eberhart and Dr. Kennedy in 1995, inspired by social behavior of bird flocking or fish schooling.

PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms (GA). The system is initialized with a population of random solutions and searches for optima by updating generations. However, unlike GA, PSO has no evolution operators such as crossover and mutation. In PSO, the potential solutions, called particles, fly through the problem space by following the current optimum particles. The detailed information will be given in following sections.

Compared to GA, the advantages of PSO are that PSO is easy to implement and there are few parameters to adjust. PSO has been successfully applied in many areas: function optimization, artificial neural network training, fuzzy system control, and other areas where GA can be applied." -Taken from Swarmintelligence.org

I only have a home PC so I have shit for computing resources, I thought that there might be a way for some of us to use a grid computing approach to increase our combined ability to find and train a program for sports prediction. They're doing that on another forum I post on to help figure out how proteins fold called folding@home it's a distributed computing project -- people from throughout the world download and run software to band together to make one of the largest supercomputers in the world.

I just don't know enough about distributed computing to make this happen

, and thought there may be someone here who does.

Mr.X,
Thanks...I try.

**MonkeyF0cker** · 09-25-09, 03:07 PM

Distributed computing is a fairly simple concept. The difficulty lies in the programming. However, with either sims or regression analysis, there really isn't a need for it. It wouldn't be worth the effort. There is far more flexibility with Monte Carlo sims than regression. Without giving too much away, simply creating a distribution for yards gained on a particular running play could have endless inputs. However, the question of noise is certainly an issue. In the end, you are generally not confined by the limitations of a single PC. I can't see the need for distributed computing in this context.

**wintermute** · 09-25-09, 07:18 PM

Karomesis

Thanks for providing a link to an actual paper using these techniques. I'll file it for future study.

I mentioned MCMC but I haven't yet been able to duplicate any of the results I've found in the literature so I presume I'm doing something wrong there.

Most of my time is spent doing Monte Carlo simulations of MLB games. These simulations use buckets of CPU time but with a little planning (make that lots) it's possible to squeeze an enormous amount of computing power out of a PC. I have a Mac and can run 135,000 game simulations a second.

Good luck

**Dave Head** · 09-25-09, 07:27 PM

Hi Karomesis

If you want to use a neural network, you can download free source code by visiting:

Annie - Artifical Neural Network Library

http://annie.sourceforge.net/

"Annie is an artificial neural network library for C++. Versions exist for both Windows and flavours of Unix (tested on Linux). The library currently has support for training, saving and executing multi-layer perceptron, radial-basis-function, kohonen maps, Hopfield and general recurrent Networks. Along with a library, also included are some example applications and binary utilities to help you construct training sets, train the network, visualise etc."

This beats the hell out of writing your own software from scratch.

IMO neural networks are not well suited for handicapping, but BOL anyway.

**Karomesis** · 09-25-09, 11:40 PM

thanks for the tip winterminute.

and thanks for the link Dave.

I have a strange feeling these methods would be well suited not to handicapping per se, but following the money.

everyone knows the sportsbooks win. But there are other people who win as well in this game without loosing their shirt, and they certainly don't follow the same guidelines as everyone else.

**Dark Horse** · 09-26-09, 01:00 PM

Typical academic pretentiousness to give an exotic name to something vague. Nothing wrong with a complex approach, but if you don't distill it into a simple essence it won't make money. Books clean up on any type of fluff factor.

**Wrecktangle** · 09-27-09, 12:55 AM

I tried a number of Neural approaches including Back Prop, I even got a Neural guy to help out. Regression was easier and better.

I'm a guy who uses a lot of tech (ask Daringly) but using tech for tech's sake will get you a lot of...uh...tech.

...I still like the idea of particle swarm...actually, I just like the name...

**Indecent** · 09-30-09, 04:44 AM

It's all in your input.... Garbage in, garbage out. How many people using neural networks really broke the stats down into something meaningful? I won't give out too much, because it was thousands of hours of work on my part and I'm not feeling that generous.

Here's a hint, if you never processed play by play stats, calculated values adjusted for opponent strength, and experimented with normalization techniques to scale the data for your network topography choice, you haven't scratched the surface of what they are capable of. It really is a bit of science and a bit of luck to stumble on network sizes, hidden layers, input scale and values, etc that work, but time, research and persistance will do you well.

**Wrecktangle** · 09-30-09, 10:41 PM

IMO there are simpler ways that are more effective...Neural is a ten-ton bitch to do right...or do wrong.

**Indecent** · 10-01-09, 03:10 AM

Originally posted by Wrecktangle

IMO there are simpler ways that are more effective...Neural is a ten-ton bitch to do right...or do wrong.

Not sure I can disagree with that. I didn't try any of the "simpler" techniques as the nueral network studies were part of research in college, but I will say that once you have a good understanding of what type of games your system is good at, I do think it's accuracy is hard to beat. The hard part is finding the way to decide which games are best, but there's a few techniques that you could use.