The free response free-for-alls

1. The first free-for-all

A while ago, Manifold user Sophia Laird made a series of self-resolving markets called, “Each option on this market will resolve to the truth of its associated statement.” Each of these markets were multiple choice markets, where each each option was a statement about the market itself, and the market would resolve to all true options at close. The final market in this series was a free response version where users could submit any option they wanted:

This one ended up being absolute chaos, since there were no rules on what to submit, aside from a request by the creator to make each option self-resolving. This led to people looking for as many exploits as possible and trying to break the market. For example, there was no rule saying that you couldn’t submit things that were guaranteed to pay out, so I submitted an answer that pays out as long as its probability is between 0 and 100%, inclusive. That one at least didn’t break the market, but some answers were specifically formulated to make it impossible to determine if they should pay out or impossible to pay out based on their truth. Examples include:

• Paradoxical options, like, “This statement is not true.”

• Options that aren’t strictly impossible but are impossible to resolve correctly because they are self-defeating, like, “This option won’t pay out.”

• The converse of the above two types, ones that could consistently pay out or not pay out, like, “This option will pay out,” or, “This option is true.”

• Options that are impossible to reasonably determine the truth of, like those asking about unknown values of the busy beaver function.

In addition to all that, there were a lot of options that weren’t self-resolving. Although the market’s description asked traders to only submit self-resolving options, the description didn’t actually say that non-self-resolving options would be excluded, so there was an ambiguity as to whether any of them would pay out. And then there were meta-options, like, “This will cash out by itself if the market creator decides it will be too much work to determine which options cash out but does not want to N/A the market.” All of this weirdness led me to create a meta-market on it:

This was the where the nickname “free response free for all” came from. I actually was originally just going to title this, “How will @SophiaLaird resolve, ‘Each choice on this market will resolve to the truth of its associated statement: FREE RESPONSE EDITION,’” but that was too long to fit in the title, so I had to come up with a shorter version. Ultimately, I thought this title was much catchier anyway.

Ultimately, Sophia Laird ending up ignoring most of the troll options (the paradoxical or impossible to determine ones), though she did come up with a clever way of getting around one of them. I made an option called, “This statement will cash out if and only if it doesn't cash out,” and that one was cashed out… at 0%.

2. Planning a sequel

Given the chaos that the first market devolved into, some of us were wondering if there was a way to create essentially the same thing, but with resolution criteria that prevent the market from being broken by certain types of options. Then we could still have chaos but of a more controlled variety. On Discord, @Joshua DMed me a list of rules that he came up with to fix the issues encountered in the market. This list was basically the first draft of the rules I ultimately used for my market. For example, the idea of including a code to identify answers or of limiting the amount of answers a user can submit originated from that list.

However, I quickly found problems with Joshua’s list. For example, the way it got around paradoxes was by stating as a rule that paradoxical statements wouldn’t pay out. This solves simple cases like, “This option won’t pay out,” but what about cases where the paradox isn’t in a single option, but multiple? The most obvious way would be to consider all options that form a set of paradoxical options to be invalid, but this could potentially allow perfectly innocent answers to be “captured” into a paradoxical set.

2. a. Tarski’s Hierarchy

I knew of a different way to deal with paradoxical options anyway, though: Just solve the associated paradox. In this case, the paradoxes were all forms of the Liar’s paradox, where a sentence declares itself false, either directly (as in the classic Liar), or in a more roundabout way (e.g. the card paradox or Quine’s paradox). Luckily, I already had an existential crisis about the Liar’s paradox once, so I know that it has a solution in formal logic devised by Alfred Tarski.

Tarski proved that no sufficiently expressive formal language can define its own truth predicate. In layman’s terms, what this means is that, if you have a language where the meaning of every expression is defined with mathematical precision (unlike natural languages where ambiguities abound), and the language is capable of expressing negation and some form of self-reference (including indirect forms of self-reference; even a language only capable of stating basic arithmetic satisfies this criterion), then there is no adjective “true” that applies to every true sentence in the language and is a part of the language itself. However, you can have a “meta-language” which contains the original language as well as an additional predicate, true₁, that meets the following condition: For any sentence S in the object language (the original language), “S” is true₁ if and only if S. This meta-language therefore defines truth for the object language, but note that true₁ only applies to sentences in the object language. “2+2=4 is true₁” is not true₁ because it isn’t even a sentence in the object language. To discuss truth in the meta-language, we need a meta-meta-language with a predicate true₂, and so on.

When we apply this to the Liar’s paradox, we see that, “This sentence is not true,” is inherently ambiguous. Which form of true are we talking about? Let’s say it’s true₁. Well, then the sentence isn’t part of the object language, since that language doesn’t have the word “true₁”, so it can’t be true₁. We can say, then, that, “This sentence is not true₁,” is true₂, since it is true in the meta-language that the sentence is not true₁. (Technically, this may depend on how the language handles category mistakes, e.g., we may say that a sentence that attributes true₁ to sentences that aren’t in the object language is simply not well-formed, rather than false, and therefore it’s negation wouldn’t be well-formed either. But I’m going to assume for this analysis that we handle them in the simpler way, by declaring any sentence that attributes a property to something that categorically can’t have that property to be false). Of course, it doesn’t actually matter which trueₙ it is. It will always be the case that “This sentence is not trueₙ,” isn’t in the nth order language, so the predicate trueₙ categorically can’t apply to it. Thus, the Liar’s paradox is solved, and we see that the version in natural language is just a result of ambiguity. The word “true” in English doesn’t have a subscript - it’s supposed to apply to the whole language, even sentences containing itself - but we already know that there is no unambiguous way to make that work. The contradictory conclusion in the Liar’s paradox results from assuming that it does and then pulling a diagonalization trick to show that it actually can’t.

2. b. My ruleset

In order to make sure my market didn’t fall prey to the same problems as the first one, I needed to have a set of rules that makes it clear what kinds of answers are valid and prevents any problems from occurring when I try to resolve it. All of the loopholes that have to do with truth could be closed by invoking Tarski’s hierarchy. I just had to explain the concept in the description and state that any time the word “true” is used without qualification, it means true₁. I also specified that the word “true” in the market description means true-ω₁ so that nobody could exploit that ambiguity. Those aren’t the only type of paradoxical answer, though. Like I said before, an answer like, “This option doesn’t pay out,” isn’t paradoxical in the same sense as the Liar’s paradox - there are no semantic problems with assigning it a truth value - but it does still have the same effect on the market, since resolving it positively would make it false, and resolving it negatively would make it true. However, I got around those types of options by specifying that valid options could only reference events before or simultaneous with the market’s closure. That ensures that any statement whose truth value depends on the resolution to the market itself is automatically invalidated, preventing that type of self-reference from posing problems either. Plus, it’s a criterion that had to be put in place anyway in order for the market to resolve.

Aside from that, I also added that any valid option had to either be something that I know or can easily find out at close. This prevents tricks like referencing unsolved mathematical conjectures that were used in the first market. Combined with my rules to avoid self-referential paradoxes, this ensured that I won’t have any cases where the market is impossible to resolve correctly.

The other rules, while not necessarily required for a correct resolution to be possible, were there to improve the quality of the market and make it clear what kinds of answers would be accepted. I explicitly stated that only self-resolving answers were valid and stated what I would do in the case of subjective or ambiguous options. I banned options that could violate Manifold’s rules, are offensive, or that just represent market types that we’re all tired of by now (like, “Will this market have X traders?”). I also stated that each user can only submit five options and that an option has to at least have a theoretical possibility of being false in order to pay out. This is to close some basic loopholes that, while not market-breaking, were already exploited in the previous market. I figured it would be boring to leave them open, since people had already seen them last time, so I closed them, while still intentionally leaving open some loopholes (e.g., you can’t say, “This option has a probability <= 100%,” but you can still say, “This option has a probability <= 99.9%”).

I also gave clear rules for how the answers should be phrased. They have to be declarative sentences and are resolved based on the truth of that sentence. The only exceptions to this are that you can add a code at the beginning so that other answers can identify yours, and I won’t be tough on typos and similar mistakes. The purpose of this was to avoid some ambiguity that occurred in the previous market. Most answers, rather than being phrased as declarative sentences to be resolved based on their truth or falsehood, were phrased as conditions under which the option should pay out. For example, “This option pays out if 1+1=2.” It would have been possible for Sophia Laird to resolve those options negatively by interpreting them not as instructions for when the option should pay out, but as declarative sentences, which she can make false by just not paying them out. That didn’t actually happen, but I figured I’d prevent any potential confusion from the start. Of course, this kind of ended up being moot because I allowed the one answer that was phrased this way anyway on the grounds that it was a mistake and was obvious what it was supposed to mean.

The one other rule was that any time the word “true” is used, it actually means “true and valid” (based on the other validity rules). This was to prevent cases where any option with the word “true” in it becomes invalid due to another invalid option. For example, if someone submitted an option referencing an unknown mathematical conjecture, then the truth values of options that reference the entire set of other options could also be unknown.

3. The almost free-for-all

Since I already used the name “free response free-for-all” to refer to the original market, I added the word “almost” to my market, to show that the chaos was more controlled.

This market ended up being my second-most popular market ever, with 107 traders, and 67 options in total were submitted (the total number of options is 68 because it also includes Other). There were also a lot of… unwise bets made on the market. A lot of people kept betting certain options up to a probability higher than 1/(expected number of options to pay out). I checked on the state of the market just before it closed, and I wish I had gone to look at it earlier, because a lot of people bet options up to way higher than the amount that each one is expected to pay out. Oh well, such is the chaos that should be expected from an (almost) free-for-all.

4. Resolution

I have the final spreadsheet that I’m using for resolution here. It will explain why each option is true, false, or invalid, and on which level of the truth hierarchy. For most options, the truth or falsehood is obvious, or it only requires a simple explanation that can fit in the spreadsheet. But for the ones that require a longer explanation, I’ll put it here.

4. a. Emergent Scrabble words

I actually misinterpreted these options at first, thinking that they were only referring to the sequence of letters in the labels. But actually, they’re just about the first letter of each option. The first letters of the first two options spell AD, which is a valid Scrabble word.

4. b. Pyramidal numbers

You can find the formula for the nth r-gonal pyramidal number here. Since the formula increases in both r and n, all you have to do to check whether a given number k is pyramidal for r,n≥3 is to loop through the sequence of pyramidal numbers for each r, going on the the next r once you reach one that’s larger than k, and terminating once you reach an r such that the 3rd r-pyamidal number is already greater than k. I wrote a Python script to do this and found that 68 is not pyramidal for any r or n.

4. c. Ten minutes to resolve

Only events before close are counted, but I did take more than ten minutes partially determining the resolution between the time this option was submitted and the close of the market. So it is valid and true.

4. d. Invalid options

I, Floris van Doorn, Bohaska, Duncn, SEE, CodeandSolder, and ShadowyZephyr all admitted to adding invalid options and not reading the rules carefully enough (Yes, even I was not careful enough with my own rules).

4. e. New York Times headline

The top option started with the letter A, but a scan of the NYT fron page from the last day of the market didn’t include any headlines that started with A. The main headline was, “Making the Planet Glisten With Gold, While Poisoning It in the Process.”

4. f. Visual errors

Here’s what the graph looked like at close (and still does unless Manifold has corrected the bug by the time you’re reading this):

The visual error is pretty obvious. For the last week (and a little before that), it shows all the probabilities as constant even though this is definitely not what happened.

4. g. Turing machines

I wouldn’t know how to determine if, “At close there is a finite-state automaton with less than P states which accepts all true options and rejects all other options, where P is the highest percentage of a single option,” is true even if P was a small number, but given that P=8, and which 8-state Turing machines halt is an unsolved problem, that makes it extra-impossible to determine. So this option is invalid.

4. h. Median character length

The median character length was 69.5. Therefore, there are no options with the median character length, so the statement about the top option with the median character length was considered false. Hopefully the current king of France isn’t angered by this resolution.

4. i. Whatever this option was

One of the options was, “This is a statement that will be true when this market resolves. As for now it is now”. I don’t know what this was supposed to mean, but even if I did, it almost surely would have been a statement that was either guaranteed to be true or guaranteed to be false, so it doesn’t pay out.

4. j. Option (W)

(W) was the one of the last options submitted and seems to have been an attempt at a troll option that would make it difficult to resolve, like the ones submitted to the original market. The option says, “This option isn't considered to have been "guaranteed to true when [it was] submitted".” This option quotes the description, so one interpretation is that true means true-ω₁, since that’s what the “true” it’s quoting meant. But of course, any sentence that has the word “true-ω₁” in it isn’t true-ω₁. The other interpretation is that it’s true₁, but the same thing applies there. One could argue, “But it’s not about whether it actually is true, but about whether it’s considered true, which is an object-level fact about your brain.” Okay, but if that’s the case, then I can exclude it on the grounds that it references something after close, since I didn’t make this determination until then. Or you could argue that it’s only about what I thought was the case before close, in which case it’s false. No matter how it’s interpreted, it isn’t true. And for an ambiguous case like this, where I’m not actually sue how to interpret it, I only need one interpretation to be false anyway.

4. k. Spelling and grammatical errors

I only counted three times when someone submitted an option with a spelling or grammatical error and stated that they had made the mistake/corrected the mistake in the comments. These were from Jai D., Odoacre, and Bohaska.

4. l. If N, then …

One option was, “At the time of close, if (N) is true then the number of responses is no more than 50.” I’m interpreting this as a material conditional. Since (N) was false, the material conditional is true.

5. Conclusion

My spreadsheet leaves me with 33 true options, so each one resolves to about 3%. This means I was being overly conservative when I was betting options down to 5%, I should have bet them down even further. I will wait a bit before resolving just in case someone finds a mistake in my spreadsheet, though. That also means you can get free mana by betting NO on this market in the next hour. I already promised not to buy NO, so I can’t get any mana from it.