Survey Results Pt. 2: Manifold on Manifold

In my last post, I went over the results to the first two sections of my Manifold survey. This post will cover Section 3, which was about people’s opinions on Manifold itself.

Who should be Trustworthy.ish?

The first question on this section was, “Which of these Manifold users do you think should be Trustworthy.ish? (Currently, none of them are yet).” That actually turned out to be not quite true by the end of the survey, though, because Pat Scott, me, and jskf all became Trustworthy.ish shortly before the survey concluded.

The winner ended up being

firstuserhere

, which I guess works out, since he is still not Trustworthy.ish yet, so the title of my market about this one won’t be a misnomer. The Manibadges markets give him a good chance of becoming Trustworthy.ish eventually, though.

@Joshua (not to be confused with @JoshuaB), who was submitted as an option a day too late to actually be on the survey, also became Trustworthy.ish. I wish I had added him myself before the market closed to see the results, but I guess it doesn’t matter now, since he officially got the badge anyway.

Another interesting thing I noticed about this question is that the lizardman constant seemed to be much higher than the others. On the previous section, only 3% of people said they weren’t human, only about 1% claimed to be from Antarctica, and most of the free response questions had few joke answers. One user noted that the lizardman constant was lower than expected. But on this section, Levi got about 7% despite the fact that he has had many controversial resolutions and was even temporarily banned for his behavior (some people even think he’s evil, but more on that later). ButtocksCocktoasten also got about 6% despite being a joke account. It could be that more people are willing to vote for joke options in the checkbox format, since they can vote for a joke option and a serious one. It could also be that the votes were genuine - maybe a few people think Levi is more tustworthy than his reputation makes him out to be, or that joke accounts can still be Trustworthy.ish as long as they resolve markets correctly.

Controversial Market Types

The second question in this section was, “For each of the following types of markets, say whether you like or dislike them. Please do not answer for any markets types that you have not personally encountered on Manifold or are not familiar with.”

The types were:

• Self-resolving: Markets that resolve based on some characteristic of the market itself, like the average percentage or the number of traders

• Whalebait: Markets that encourage large bets because a user with lots of mana can more easily influence their resolution

• Personal: Markets that resolve according to some aspect/event in the creator's personal life

• Subjective: Markets that resolve based on the creator's opinion on something

• Meme/joke: Markets that were made for a meme or joke rather than serious predictions

• Death: Markets about whether a given person will die by a certain date

• AI wiping out humanity: Markets about whether AI will wipe out humanity

• Parodies of AI wiping out humanity: Silly variations of AI wiping out humanity, like "Will AI wipe out AI?" or "Will AI out-wipe humanity?"

• Rationalussy: Markets about the made-up word "rationalussy"

• Poll-based: Markets that use a poll to determine the resolution

• Stock: Markets that never resolve and are meant to reflect how much people like or dislike the subject

• Far future: Markets that won't resolve until a very distant time

• Long odds: Markets on things that are basically certain to happen or not to happen, and for which everyone agrees that this is the case

• Daily: Series of markets that are about the exact same commonplace occurrence, but shifted back by a short period of time, e.g., a series of markets asking "Will Stock X be above price Y on date Z?", where there is a new market for every day.

• "Completely legal Ponzi schemes": Bountied questions where the first people to add to the bounty get mana from people who add to the bounty later, but are completely open about being Ponzi schemes

Here is the graph from Google forms. You can probably already tell that personal markets were the most-liked, and whalebait was the least-liked.

Interestingly, death markets, which people expected to be a lot more controversial based on my market on the subject, actually have the fewest dislikes out of any option. I was also surprised that Ponzi schemes were disliked on net, since most people who participated in them seemed to like them (I thought they were very fun, especially Code and Solder’s original one). But there was one user who made rug-pull versions, so that may have soured people’s opinions on them.

Based on the number of net likes, here is the list of controversial market types, from most-liked to least liked (using net proportional likes as a tiebreaker). The parentheses after the net likes on each option are the total numbers of (like, neutral, dislike) responses for each type.

1. Personal: 65 net likes (87, 51, 22)

2. Death: 29 net likes (50, 87, 21)

3. Poll-based: 17 net likes (54, 65, 37)

4. Meme/joke: 12 net likes (57, 54, 45)

5. Long odds: 10 net likes (46, 74, 36)

6. Far future: 10 net likes (50, 69, 40)

7. Daily: 8 net dislikes (36, 74, 44)

8. Subjective: 11 net dislikes (34, 79, 45)

9. AI wiping out humanity: 21 net dislikes (37, 64, 58)

10. Completely legal Ponzi schemes: 27 net dislikes (32, 61, 59)

11. Parodies of AI wiping out humanity: 36 net dislikes (38, 45, 74)

12. Self-resolving: 53 net dislikes (27, 48, 80)

13. Rationalussy: 55 net dislikes (29, 43, 84)

14. Stock: 58 net dislikes (21, 58, 79)

15. Whalebait: 70 net dislikes (16, 45, 86)

I guess I am a bit of a contrarian, because I liked a lot of the types that are at the bottom of the list, though I agree with the dislike of stocks. But this also isn’t too surprising (except for death markets being the second-most liked), since I already thought whalebait was most likely to get the lowest rating.

I wish someone had done a survey similar to this one in the past. It would be interesting to look at the longitudinal results and see how the popularity of different market types has changed over time. For example, I suspect that self-resolving and whalebait markets took a big dip in popularity after the Whales vs. Minnows drama, and rationalussy markets probably weren’t so disliked before they became oversaturated. Stocks also seem like they’ve become less popular over time, since I see them way less often than I used to.

What types of markets have users created?

I asked respondents what markets types they had created, out of all the types that I currently know about.

While the vast majority have created a market, the only market type that most people have created is Yes or No.

I was kind of surprised that more people had created some of the deprecated types than the current ones, especially the current version of free response. Overall, there were 60 respondents (36.8%) who had created at least one deprecated type, almost as many as there were who had created the current version of multiple choice markets.

(And of course, by current version, I mean the current linked version, since unlinked multiple choice didn’t come out until after the survey closed).

Incorrect and Controversial Resolutions

I asked the respondents if they had ever misresolved a market, with the clarification that, “It doesn't have to be an intentional misresolution. A resolution that you believed was correct when you made it but now believe was incorrect, or one that you made accidentally would count. It also counts even if the resolution was later changed.” This one I actually had to change my own answer to because I accidentally misresolved my Cover’s paradox market while the survey was still open (the resolution is fixed now, though).

Only 12 respondents said that they had ever misresolved a market, and 8 said they weren’t sure. The other 132 said no. Intuitively, I thought that there would be more people who had misresolved a market than this, but this was likely a result of the availability heuristic - the people I think of as the “typical” Manifold users because I see them the most often are actually the power users who have created lots of markets and are therefore likely to have misresolved some. The real average Manifold user has not created that many markets, and certainly nowhere near as many as I have, so it makes sense that the vast majority haven’t misresolved a market.

I also asked a question about controversial resolutions, with a description clarifying, “Have you ever had a resolution that caused significant controversy among traders, even if you believed it was correct?” These were the results:

In total, 17 people have had a controversial resolution, 128 haven’t, and 6 aren’t sure. As we would expect, there are more people who have had a controversial resolution than a full-on misresolution. The slice of people who weren’t sure was also smaller, probably because part of that, “I’m not sure,” slice from the first question was from people who had controversial resolutions that even they were not sure if they had done correctly.

Is Levi evil?

I asked a question about one of the the more controversial figures on Manifold, Levi Finkelstein (the description of the question specified that this is who “Levi” refers to). Although Manifolders don’t think he should be Trustworthy.ish, the majority agreed that he is not evil, in a 65 - 43 vote.

Can a cabbage last 53 days?

I intentionally asked this with as little context as possible because I thought it was more fun that way. The only context given was, “This is a matter of significant controversy on Manifold.” For those not in the know, it was a reference to the infamous Prigozhin cabbage market:

Ultimately, the cabbage in that market lasted 53 days, and it might have lasted even longer, but Quantum Gambler ate it after the reports of Prigozhin’s plane crash, since it was no longer necessary to keep the cabbage after it had already outlasted Prigozhin.

However, some people were suspicious of the market because they didn’t believe that a cabbage could last as long as this one did, and because the creator wouldn’t post any pictures of it in the comments, despite many people requesting them. Ultimately we did get pictures, but only after Prigozhin had already died.

We also got confirmation that the cabbage still tasted good:

However, some were not satisfied with this and didn’t trust the creator’s word about the cabbage’s longevity. Some wanted video evidence.

Others decided to perform experiments for themselves to see if a cabbage could really last as long as Quantum Gambler said it did.

In light of this controversy, I decided to make this question to see what most Manifolders thought of the issue. The results of the survey show a vindication for the Prigozhin cabbage: 81 people agreed that a cabbage can last 53 days, while only 47 people didn’t.

If you still haven’t had enough cabbage business after all this, you can check out the Cabbage topic on Manifold. There are currently hundreds of polls about cabbage to participate in.

Manifold events

I’ve mentioned before that there have been four markets on Manifold that felt like events in and of themselves. At the time, it seemed like they had almost consumed the whole site, such that they were the main thing everyone was talking about.

Those markets were “The Market”, Whales vs. Minnows, the AI risk letter, and LK-99. That last one in particular brought a lot of new users to the site, so I was interested to see how many users were new and how many had been around since the old days of 6 months ago. I also added in a few other events for even earlier users: the 2022 midterms (which I think is the event that caused me to become a regular Manifold user), and Manifold actually becoming Manifold (it used to be called Mantic Markets).

Here are the results:

(The options with ellipses are “I remember when it was still called Mantic Markets” and “I haven’t witnessed any of these”).

Based on these results, it looks like the median Manifold user is someone who first joined during Whales vs. Minnows, since a slight majority were here to witness WvM and the AI risk letter market, but all the older events have <50%. This makes me feel like a real old-timer, even though the oldest event I was here for is still less than a year ago.

Interestingly, more people remember Whales vs. Minnows than the AI risk letter market, even though Whales vs. Minnows happened earlier. This is probably because it got a lot more attention, going by the number of traders (although that was in part because the AI letter market was unlisted). So although I perceived the AI risk letter market as a big event, on par with “The Market” and Whales vs. Minnows, it might not have been as big to the average user. Oh well, maybe my perception is biased because I lost 8,000 mana on it.

Another interesting thing I noticed is that there’s a large number of people who were here for Whales vs. Minnows but not “The Market”, even though they happened almost directly after one another. This is probably because a lot of those people specifically joined because of WvM. So at least some of the new users who joined because of that market decided to stick around.

Does Manifold have too much of a focus on AI or niche tech topics?

Manifold has a disproportionate number of markets about tech, which isn’t too surprising given that it’s a tech startup itself. And of course, the strong ties to the rationalist movement mean that there are lots of computer geeks, AI enthusiasts, and AI doomers here. But is it too much? I could certainly see a lot of people who think Manifold kind of sounds interesting looking into it, seeing all the tech-based markets, and deciding it’s not for them. This could even be partly responsible for Manifold’s uneven gender ratio that was revealed in the last post.

This ended up being one of the closest questions on the survey, but in the end, No slightly edged out Yes, 82 votes to 77.

So Manifolders are very divided on whether the strong focus on AI and tech is a problem, with a slim (though probably not statistically significant) majority saying it isn’t.

Favorite Markets

I asked users, “What is your favorite market on Manifold, if you have one?” Here are the responses:

GDP in 2028 if a democrat gets elected 2024?

 

"The Market"

"The Market" - I had the most fun with it and the variants it spawned

Do most Manifolders believe a cabbage can last 53 days?

Football (round ball) ones, Australian politics

Hanging Paradox

How will [UK] house prices change in the 12 months to December '23?

I appreciate the sheer audacity of: https://manifold.markets/JosephNoonan/conditional-on-there-being-a-ball-p

I enjoyed the Ponzi market (though I didn't participate, and don't want many more of them)

In 2028, will an AI be able to generate a full high-quality movie to a prompt? (free money)

LK-99

LK-99 

LK99

LK99 during the first weeks

Maybe the moral realism one?

Mine

My marriage market ofc

My own market about whether Infowars will go down by 2025.

None yet

Ponzi

Really liked the Oscars 2023 markets

Snake Eyes

Sports

Sudoku

That one about generating movies via AI in 2028

The jigsaw puzzle markets

The lk-99 one was good

The one about Bukele

The one about which set of points is human-generated

There's too many markets I like well enough.

Ukraine

Vegan pizza

Whales vs Minnows? 

Whales vs minnows

Who will win the 2023 speed chess Championship 

Will I get any rationalussy at Manifest?

Will Jimmy Carter become a Centernarian

Will Putin attend BRICS?

Will ai wipeout humanity (skateboard)

Will anyone draw something recognizable on this market graph?

Will carti drop :(

WvM 

Yudkowsky's "If smarter than human AI has an okay outcome, what will be the reason?"

Zen Stock

ai movie 2028 market

don't you dare make me choose among my children

firstuserhere

https://manifold.markets/Loppukilpailija/which-of-these-random-data-points-i

https://manifold.markets/Mira/will-a-prompt-that-enables-gpt4-to

https://manifold.markets/NickAllen/this-market-will-resolve-yes-on-apr?r=QXJ1bkNoZWJyb2x1

https://manifold.markets/QuantumObserver/will-the-lk99-room-temp-ambient-pre#qI936RhIH16LOihKdLmP

https://manifold.markets/Quate/will-the-a-press-count-for-super-ma-994cbd8ef908

https://manifold.markets/ScottAlexander/in-2028-will-an-ai-be-able-to-gener

lk-99

markets made by good users

political outcome events, like coup in Niger

sex + love

the LK-99 market

whales vs minnows

which random point set is human

“Will Oppenheimer and Barbie perform historically well at the domestic box office on opening weekend?”

The empty space in between the first two is because someone just submitted “ “ as an option.

The most popular market was LK-99, which makes sense since it also has the most traders of any market. Whales vs. Minnows was also pretty popular, as were the market on whether AI will be able to generate a movie by 2028, and Which random point set is human generated?

What features should Manifold add?

The next question aksed, “Are there any features that you wish Manifold would add?” Here were all of Manifold’s desired features:

1) A default @ to ping the market creator 2) the ability to filter trade history on markets by date and/or size

A market type to predict dates of events

A numerical market type, but like e.g. Metaculus not just for the expected value

A search function that works

A way to almost completely hide markets with a close date more than a year in the future. And a way to opt-out of leagues

An option to short 

Automatically calculate selling position at a certain percentage - the "Sell position at the minimum % with no loss" button. Also a Notification when a market moves by a certain %, James promised to add that like a month ago and forgot. How many people set up 1 mana limit orders for this? it's clearly something The People want.

Be able to alter your banners on platform!

Better mobile app

Better question discovery, but not social algorithms, more like better filters and categories

Better search functionnalities. Better ways to sort markets

Charity leaderboards

Comparing my profit to others in a group, seeing who holds what positions in a multiple-choice market

Conditional markets (double-bet Mana because one of them resolves NA anyways), more advanced filtering options (e.g. certain percentage range plus minimum number of traders).

Decision Markets

Different mana system for long-term markets. As of now there's little incentive to bet in markets that solve in the distant future.

Early resolve a multiple choice/free response answer to NO. This would enable longer running markets that continuously add new possible answers without the UI getting cluttered.

Filtering of the markets included in the profit graph by search phrases, tags, etc.

Hidden stop loss orders

I find it hard to bookmark questions and then find them

I have an idea for a certain thing that I’d like to exist on Manifold; it’s too big to explain here and arguably could be implemented with current functionality (but not optimally.)

I made a comment but let's say DM for short

I'd rather them prioritize a fast bug-free user-friendly experience than new features these days, I think. But there are probably some new features I'd like added if I thought about it

Let me see percentages smaller than integers, as an option! I know decimals! It's fine!

Live sports scores / instantly resolving sports markets

Manifold Monarch

Market liquidity graph

Markets that predict a date! 

Moderation

More users

NSFW section (idk what this would do)

No

Not willing to share, I'd create my own PR to try and add them.

Partnership with rationalist-associated shops to buy real goods with mana.

Pin comments, reacts to markets and comments

Positions on multiple choice

Private groups

Private groups with independent currency

Public tips, multiple choice adding up to more than 100

Random numbers generation that can be attached to any market. The number is shown as a hash before the creator pushes a button to reveal it

Revert the permanent stock change (or at least ban bots from them and allow limit orders)

So many! If I had to choose one: ability to sort portfolio markets by close date on mobile.

Something to encourage more trading on socially valuable markets relative to less socially valuable ones

Subgroups/subcategories, ability to merge groups

The ability to block certain types of market, like stock markets

There's lots of low effort , stupid markets that get made and I could use better ways to filter those out. Most recently the JOSE LUIS RICON' stuff where I just ended up blocking the user who was making them, which wasn't ideal because that guy sometimes makes markets I would like to see. I don't know of any ways to do this besides making it more difficult to make markets, which doesn't seem acceptable.

Tons

Too many to list

Unlinked multiple-choice markets. Please. I have SO MANY markets I want to make that WON'T WORK AT ALL under the current system and it is FRUSTRATING and it makes me type in ALL CAPS

Website login for Apple ID users

What happened to that community game market from a while back? Add battleship so I can win

Yes, mainly the ones I have already made markets on

You can use mana to advertise your market

ability to customize the order of multiple choice answers

anonymous surveys+changing votes

better curation and organization of high-quality markets

better limit order/options etc

bookmark list

dms for the creepy crawlers

more badges for all sorts of things that even users can transfer to other users

selling mana for crypto

tips

web notifs for markets people you follow create!  price change notifs! separate tabs on the notif page for different types! lots and lost more.

yes

There was a big variety of suggestions, so it’s hard to find that much of a pattern in them. But a lot of people seem to want some features to help them find markets more easily. One of the most popular suggestions was unlinked multiple choice markets, which is no longer a suggestion - it was just implemented yesterday. So at least Manifold is giving the users what they want in this case.

Should Manifold bring back the big text?

This was another somewhat cryptic question (what can I say, I like making questions that will seem mysterious the people and maybe make them curious about what they’re asking). It told repondents to only answer if they know what the big text is. Evidently, most people did not, since only 46 users answered the question. The big text refers to an old feature that was originally implemented as a joke, where bets in the trade history would be in a larger font the more mana was invested. This went to comedic lengths, with 10,000 mana bets being enormous. The feature was eventually removed, but many people missed it. Even though it was not a “serious” feature, it did make the site more fun.

Out of the people who knew about it on my survey, there was a strong consensus that the big text should return:

There was, however, disagreement over whether it should be an optional feature or just be turned on for everyone. This surprised me, since I thought most would agree that it should be togglable so that people who find it annoying can just turn it off.

In total, the vote counts were

Yes: 22

Only as an optional feature: 16

No: 8

The Manifold Experience

The next question was, “How would you rate your experience on Manifold, on a scale of 1 to 10?” As you can see, most people rated their experiences as positive:

This is not too surprising. After all, why would they still be using the site if their experience was strongly negative? There were a few respondents who had a bad experience, though.

Overall, the average rating was 7.51. Manifolders are happy with their experience, but not super happy. Though I imagine that this is better than the results we would get on some other social media sites (now I wish I had done a comparison).

Is there anything Manifold can do to improve this number? Obviously, implementing some of the suggestions above might make people more happy with the site. But I also asked one more question about how Manifold can improve as a site:

What don’t people like about Manifold?

The last question on this section was, “Is there anything you don't like about Manifold?” Here are all the answers:

"Culture of contempt"-fueling markets

@johnleoks

As a new user, it's hard to find markets where I feel I have an edge, and I'm not trading coinflips/news. 

Buggy

Can be a bit laggy on my old phone (but this has improved over time)

Can't find markets I'm confident about most of the time.

Casino-like-ness on weird shit (rationalussy, keynesian beauty contests). Self-resolving galaxy-brained quant battlegrounds, on the other hand, are interesting. Is there are fine line between the two? Yes.

Deceptive resolution criteria

Eh mostly the specific markets tend to be kinda boring

Hard to find fresh interesting questions 

Have to block too many people.  Can't see comments from blocked users.

Should be able to:
Block seeing people's markets, and have that be independent from:
blocking their comments

Blocking their comments shouldn't hide them completely.  They should just be collapsed by default with the ability to expand them, and also see responses to their comments from other users

I should still be able to see a user's profile page even if I have blocked them

I don't like that I suck

I don't like the recent increase in low-quality and not-useful markets (like "Is Barack Obama Gay?", "Daily coinflip", arguably daily S&P/crypto markets, arguably daily late flight markets).

I don't think most people get my zoomer Twitteresque humour

I lost access to my old email and there appears to be no way to shift my account over.

I lost mana

It is a bit *too* addictive.

It's a bit buggy and can be slow.

It's addictive and the value of the time invested seems nonzero, but otherwise very unclear. 

It's blocked on my work network, probably automatically, and probably due to having .markets in the URL. Metaculus is not blocked.

It's pretty good overall!  I think it would be cool to continue clearly focusing on "making people bet as easily and as much as possible" as a way to get them involved, rather than occasionally slipping into "it's not good to make it easy to bet, because maybe dumb/uninterested/new/unqualified people might vote".  I still think max ease of voting is the key growth feature, so people then allow you to notify them about interesting stuff with strong context

I’ve grown to hate self referential markets

Lack of moderation

Laggy app, all the market types I put "dislike" for above, difficulty to find new markets, bugs.

Leagues encourage too much short-term betting

Lots of features were put in the about page in the current UI wich doesn't make sense

Massively US-centred userbase. 

Metaculus is more collaborative and people share information about their predictions. Manifold is more competitive, so people are not incentivized to explain their bets honestly. No idea how to fix this though.

Mobile app / PWA is a bit janky

Needs more diverse markets!

No good standard method for dealing with markets being distorted by AI risk. Like many markets I essentially have to do 'conditional on no unsafe AGI' which puts us in odd worlds. 

Not enough interesting markets

Not enough mana :)

Not yet.

Often make limit buys in bulk, and it takes ages. Maybe some quick patterns or something would be good. The live view is good, but it doesn’t seem like the other tabs on the live view are actually live.

Overall I'm pretty satisfied. I think their biggest challenge is probably not sounding like a crypto / not feeling untrustworthy due to the gambling association / just marketing to a broader audience. Idk.

Overwhelmed by jokey / personal / not interesting questions

Replacement of leaderboards with leagues

Site going down/technical issues a bit to often, inconsistent and a bit to relaxed moderation, to much mana given away (!), to many low quality markets allowed that are to cheap to make (including half of mine).

Slow moderation

Small userbase

Some things I find kind of ugly:
- the blue outline as a hover effect on question cards
- the out of proportion crane logo
- not a fan of the indigo theme color. Very "default"

Sometimes people can be unpleasant/unfriendly. Not sure what can be done about this while keeping an open intellectual culture though.

Spam questions

The "for you" section doesnt have enough new markets.

Also id like to bet on the news more

The bro-ey vibe. Many of the admin decisions.

The rules are wishy-washy, which sortof makes sense because they're not sure what they should be yet, but it leaves uncertainty for anyone getting anywhere near the very fuzzy boundaries. #justiceforlevi

The search function is still very bad.

The very existence of Manifold

There's no back button on the app if you go to a user's profile from the positions tab on a market

There's no way to make money on a market unless it's misvalued.

Time lag between betting in markets and updating  profit/net worth balance on my profile. Different profit displayed on profile/when viewing a given market directly. 

Too few markets related to my local context

Too many markets hinge on matters of definition, rather than being interesting questions.

Too many tech bros

Too much low-quality markets, bad incentives for long-term markets.

Too rationalist, we all kinda look like weirdos

UI not to my taste 

Yes

Yes

Yes.

bugs

comments get deleted when I lose internet connection for a short time, it causes addiction (takes away time), it incentivises people to start new accounts because of the irreversible effects of bad predictions on your calibration

duplicate and/or low-quality markets (outside of properly tagged joke/meme markets)

it's not enjoyable enough to make up for how compelling it is

leagues are kind of bad

probability graph axis should always go from 0-100, why are they sometimes cropped????

some people are annoying

that it's not used for more things and part of internet culture

there is a quest to reply to old markets but I cannot conveniently find them

too much focus on community drama

uses too much Internet traffic

yes

The most common issue seemed to be that the site has too many bugs or too much lag. I suppose that’s a good thing, since it’s a problem that can, in principle, be fixed. A lot of people also didn’t like the types of markets they saw on Manifold, either finding them too boring or too jokey. This seems harder to balance, since there are also a lot of people who do like those markets, but maybe if there were more ways for people to select what kinds of markets they see, fewer people would have complaints about this. People were also concerned about the moderation, the addictive features (especially Leagues), and some of the community’s behavior.

That’s it for this post. The next one will be on either Manifold’s favorites or the philosophy section, depending on which I finish analyzing first.