Fairpool is a DEX for personal tokens. Create your own token & receive royalties from trading volume. Increase volume by offering dividends / selling content for your token. Subscribe to our Telegram / Twitter to get notified about updates.
Weekly progress
- ⚒ Worked on our decision-making system.
Differences between the game-like environments and the real world
We’re working on our decision-making system. One of the ways to make decisions is to train a neural network to do it for you. Sounds great in theory, let’s see how it works in reality.
When we talk about “neural networks” nowadays, most people think about ChatGPT and Midjourney. However, these are “generative neural networks”, not “decision-making neural networks”. In other words, these networks are trained to generate new content which is similar to the training content. That means such networks can only generate new strategies if there is a training set of existing strategies. Furthermore, the generated new strategies would be similar to existing strategies.
So instead of looking at “generative neural networks”, we should be looking at “decision-making neural networks”. Such networks produce actions (= make decisions) according to their perception of the environment.
To train such a network, we need a model of the environment. More technically, we need a `State` type and a `State -> State` function. This function needs to call each agent’s decision-making function. The decision-making function for each agent has the following signature: `(agent : Agent) -> (belief : Belief agent) -> Action belief`. Here,`Belief agent` is a partial representation of the `State` available to a specific agent. Also, `Action belief` is an action that is returned with respect to the current `belief` (a concrete element of the `Belief` type, passed as the first argument). Note that each agent has their own `Belief agent` type (it might be either the same or different from other agents). Correspondingly, each agent takes their own `Action belief` (for example: a bird has access to `fly` action, but a dog doesn’t).
However, two problems arise:
- We don’t know how exactly the world works (technically: we don’t know the exact definition of `evolve : State -> State`).
- We don’t know how exactly other agents behave (technically: we don’t know the exact `decide : (agent : Agent) -> (belief : Belief agent) -> Action belief`).
While we don’t know the exact definitions, we do have a list of data points: a list of different states, a list of hypotheses for some pairs `a b : State` that `a.timestamp < b.timestamp`, and a hypothesis that all states are connected through a single `evolve : State -> State` function. In other words, we hypothesize that applying `evolve` to `a` multiple times gives `b`. Technically: for each two states `a b : State` where `a.timestamp < b.timestamp`, there exists `n : Nat` such that `evolve.run a n = b`
A simple example would be a `State` with two fields: `temperature` and `timestamp`. Such a state represents a correlation between the temperature and the timestamp, assuming that all other variables are fixed (same location, same rotation, same everything else). If we measure the temperature multiple times at different timestamps, we will have a list of such states. This list can be plotted on a 2D surface (e.g. screen or paper). Plotting the data directly will only give us some points, not the full line. We hypothesize that all states are connected by a line that smoothly interpolates between two adjacent points. This line will be the plot of `evolve : State -> State` function (inferred from limited data). Sometimes, this line will exhibit periodic patterns. For example, we’ll see that the temperature at 8am was lower than at 1pm. We can assume that `evolve` is periodic, and thus extrapolate it beyond the existing data set (simply copy-pasting the period).
Important: every prediction of the future relies on the hypothesis that either `evolve` or its nth derivative is periodic. It is impossible to extrapolate the function without the assumption of extended periodicity. In fact, every definition of the form `y = f(x)` is an assumption that `y` can be predicted from `x`, which implies extended periodicity
Note that `evolve` itself doesn’t need to be periodic — only its nth derivative needs to be periodic. For example, `y = 2 * x` is not periodic, but its 1st derivative `y’ = 2` is periodic. That means we might not see any patterns in the `evolve` function immediately, but notice them in the nth derivative.
Another important thing is that “data points” are not really points. In fact, they are probability distributions, because every measurement is erratic. So when we write `{ temperature := ⟨30.0, .celsius⟩, timestamp := ⟨1695013260327, .millisecond⟩ }`, we actually mean that the temperature was log-normally distributed with a mode of 30.0 degrees on the Celsius scale while the timestamp was normally distributed with a mode of 1695013260327 milliseconds. So technically, our list of data points can be described by any function. However, every function has a specific probability of being the “true” description. For most functions, this probability goes down very fast with the addition of new data points, because the probability is multiplicative: the function has to match all data points, so adding 1 data point means adding 1 factor to the function’s probability formula.
Overcoming the difficulties
In the previous section, we have shown that it is hard to define the exact state evolution function and the exact agent decision function for external agents. How do we overcome these difficulties?
One way is to concentrate our bets on the environment that is already defined — in other words, on a program with known code (or at least with behavior governed by known laws). This is exactly what a blockchain is. It seems pretty straightforward to train a neural network that would find a winning strategy for this environment. Note that even if the environment is defined, the behavior of other agents is still not defined. This can be solved with prioritized self-play. An extra bonus of this approach is that the resulting policy will be robust against changes in the behavior of other agents (i.e. different market conditions).
Another way is to collect a lot of data and infer the necessary functions. This seems harder — it’s exactly what every scientist does.
Another way is to assume equivalence between yourself and a sizable set of other people. With this assumption, you can just build the products for yourself and share them with other people. Incidentally, this is how most successful startups have started.
Next week’s focus
- ⚒ Develop our decision-making system.
About Fairpool
Fairpool is a DEX for personal tokens. Create your own token & receive royalties from trading volume. Increase volume by offering dividends / selling content for your token. If you want to get notified about updates to our products, please follow our Telegram & Twitter.
Any questions? Reach out to us:
Website: fairpool.io
Telegram: @FairpoolDEX
Twitter: @FairpoolDEX
Anchor Podcasts: @FairpoolDEX
Medium: fairpool-dex.medium.com
$FAIR token: Uniswap
