Dec/090
Liquid State Machine Papers
I've done a bit with echo state networks (ESNs) and mostly neglected the more neuroscience motivated liquid state machine (LSM) research. However, there is some great work going on led by W. Maass. Here are two stand-out publications:
Distributed Fading Memory for Stimulus Properties in the Primary Visual Cortex - actual data from cat neocortex showing reservoir computing sort of responses.
State-dependent computations: spatiotemporal processing in cortical networks - fascinating review of a large body of research including nice discussions of high-dimensional trajectories.
Dec/090
Focused Papers
The goal of most research papers is to focus on a particular objective or hypothesis (survey papers being the major, valid exception), but most papers have lots of extraneous details coming from the exact system and configuration under study.
After discussing the goal of boiling down a problem to its minimal required complexity with Phil Bones and Allan McInnes, I was reminded of the following paper comparing selection mechanism in GA. The point of the paper contains some survey-like discussion, but I admire the simplicity of the system they study. The system choice makes the selection behavior easy to illustrate without extra complexity.
A Game-Theoretic Investigation of Selection Methods Used in Evolutionary Algorithms -- Ficici, Melnik, and Pollack
Oct/090
Vorpal Slot Definitions
Vorpal now has special syntax to streamline slot definition. I hope writing clear Vorpal code will be easier now. In the past, Vorpal required a highly manual form of slot definition:
x = new() x.a = new() x.a.b = 2 x.a[0] = 4
No longer! Now the same result can be achieved with a nicer syntax (and slightly more efficient execution):
x = new(){
'a' = new(){
'b' = 2,
4,
},
}
Oct/090
Langford on Research
Here's a nice posting from John Lagford on pursuing grand ML challenges.
Necessary and Sufficient Research
Researchers are typically confronted with big problems that they have no idea how to solve. In trying to come up with a solution, a natural approach is to decompose the big problem into a set of subproblems whose solution yields a solution to the larger problem. This approach can go wrong in several ways.
- Decomposition failure. The solution to the decomposition does not in fact yield a solution to the overall problem.
- Artificial hardness. The subproblems created are sufficient if solved to solve the overall problem, but they are harder than necessary.
Then later he lists some things he thinks are necessary to achieve, "a master machine learning algorithm that can solve any reasonable learning problem where “reasonable” includes at least the set that humans can solve."
- Large data
- Online learning
- Interactive learning
- Compositional design
- Large contexts
- Nonlinearity
Read the post to see what these qualities mean in detail. I agree with all of them, but I haven't done a lot to pursue #1 yet.
Sep/092
Vorpal Labeled Arguments Proposal
Here's a good discussion of how Python's named parameters and Objective-C's interspersed arguments differ:
The named parameters of Python are nice on occasion, because they allow you to specify a few out of many optional arguments, but the major benefit is that arguments can be labeled in either system. I'm sure we've all seen code like the following:
image.show(xPosition, yPosition, -1, true);
What do the last two arguments mean? Of course we could clarify by adding comments or intermediate names:
int offsetBetweenPixels = -1; bool useOpacity = true; image.show(xPosition, yPosition, offsetBetweenPixels, useOpacity); image.show(xPosition, yPosition, -1 /*offsetBetweenPixels*/, true /*useOpacity*/);
Vorpal could easily support labeled arguments. The simplest implementation would take labels ending in a colon and make them part of the method name.
Sep/094
Iterated Function Systems
Fractal Flame iterated function systems are fascinating. I've always though a simulated robot that was the iterated particle would make for a good RL problem. Basically, the robot would be iterated in space by the IFS and it would need to learn to navigate to target locations using small forces (think of it as a waterbug in a swirling pool).
The source in C++ (requires libgd). Here's an example of an IFS generated from a billion samples.
Sep/092
Intro to Hierarchical RL
Hierarchical reinforcement learning is a current interest of mine. Specifically, I thin there's progress yet to be made on predictive hierarchical systems. What's a good set of papers to get introduced to these topics? Here's one suggestion:
Introduction:
A nice survey of RL (1996)
Barto's Intro Lecture
Hierarchical RL:
Feudal RL (1993) -- a nice place to start, good descriptions
MAXQ Learning (1998) -- an advancement on Feudal because the hierarchy forms a global value function. From the abstract: 'The paper defines a hierarchical Q learning algorithm, proves its convergence, and shows experimentally that it can learn much faster than ordinary “flat” Q learning.'
Prediction:
Predictive State Representations -- well presented, though a bit abstract
Aug/093
Concurrent Vorpal
Investigating and implementing concurrency in Vorpal has been going well. The idea is to introduce an message based (actor-style) concurrency. Many Vorpal processes can send messages to each other and the VM running those instances of Vorpal are run in a specified number of OS threads. Because each VM is well isolated, there have been remarkably few concurrency bugs so far.
Aug/093
Compression and Prediction
An investigation of compression software for the Mac, led me to discover work on Prediction by Partial Matching which is highly similar to the system that Peter Raffensperger has discussed here on CDA. The link above contain references to interesting papers. Also linked to it is the massively complex PAQ family of algorithms that have recently won the Hutter Prize (itself an interesting exercise) for compression improvements to text.
Prediction by Partial Matching (PPM) is an adaptive statistical data compression technique based on context modeling and prediction. PPM models use a set of previous symbols in the uncompressed symbol stream to predict the next symbol in the stream.Predictions are usually reduced to symbol rankings. The number of previous symbols, n, determines the order of the PPM model which is denoted as PPM(n). Unbounded variants where the context has no length limitations also exist and are denoted asPPM*. If no prediction can be made based on all n context symbols a prediction is attempted with n-1 symbols. This process is repeated until a match is found or no more symbols remain in context. At that point a fixed prediction is made. This process is complementary to that followed by Dynamic Markov Compression (DMC) which builds up from a zero-order mode.