Plasma's Manifold Survey is out!
For the last month, I’ve been making markets about my upcoming Manifold survey. At long last, the survey is finally out. So far, 102 people have taken it already, and you can take it here: https://forms.gle/xZqWVxuY5irgLigu9
I’ll be keeping it open through Oct. 7. As far as I know, there isn’t a way to set up Google forms to stop accepting responses at a predefined time, so I’ll just have to manually do it some time on the 8th. At that point, we’ll get to find out about the results. You can bet on what the results will be in the Plasma’s Manifold Survey category on Manifold.
1. Origin of the survey
This isn’t the first survey I’ve run on Manifold, but all the previous ones were only on a single topic. I actually ended up repeating one of those (the best dwarf planet question) on my new survey. These previous surveys ended up being good practice for this one - they taught me to be very careful about the settings I use on Google forms, since it turns out that Google forms defaults to allowing multiple responses. Why anyone at Google thought this was a good idea is beyond me.
I first got the idea to run a big multi-topic survey after seeing Pat’s Manifold Survey, which mine is sort of a sequel to. I already had some ideas floating around in my head to potentially make a future survey on, but my surveys usually didn’t get many responses, since they were generally only responded to by the people who happened to stumble upon one particular market. Making a survey like Pat’s seemed like a natural response to that problem.
For a while, I was just bouncing ideas around in my head for what to put on my survey, but I was finally spurred to action after Conflux made a market asking whether anyone would bring a ball pit to Manifest. Although I didn’t have a fully-formed idea of what the survey would be yet, the need to make a well-timed joke won out over any sense of caution, and I made the first market about my survey, “What would you do for an extra hour in the Manifest ball pit?”
After that, I just kept coming up with more and more things to put on the survey, so much so that it may have gotten a little out of hand. I’ve made more than 80 markets on the survey already and will probably make some more before the final results are in. So far, my survey has already been enough to get me onto the all-time creator leaderboard and get me a new most popular market.
Although I could have continued adding questions to it basically forever (I’m sure that by tomorrow, I’ll have five new ideas that I can’t believe I didn’t think to include EDIT: I haven’t even finished writing this, and I already thought of more), at some point the survey had to actually come out so that people could take it. I didn’t want to make people betting on the markets wait too long, and I didn’t want the survey to become too bloated (both as a result of me adding questions and other people adding options to the free response markets), so I decided to set a private deadline for myself to release it in the first week of September, and to motivate myself to actually get it done by that time, I created a market on when I would release it. I ultimately ended up being pretty busy that week, so I didn’t get it done until half an hour before my deadline, but hey, it still counts.
2. What’s on the survey?
The survey has six sections. Well, six normal sections, that is (more on that later). The first section is just an introduction with only one silly question, and the rest of the survey contains a mixture of serious, silly, and semi-serious questions. The serious questions are mostly either demographics, questions about Manifold itself, or questions about philosophy.
One thing I was interested in, aside from just the general information about Manifolders and their opinions, was the connection between Manifold and other subcultures. It’s no secret that Manifold is popular in the online rationalist community (i.e., the subculture of sites like LessWrong and Astral Codex) and that a large portion of its user base is either linked to that community, works in tech or tech-enthusiasm, or both. But I wanted to find out just how big that proportion was, and whether users not linked to that subculture felt left out. The main question I asked for this was one that just asks whether the respondent is part of the rationalist community, with options allowing them to express various degrees of involvement.
I also had a question about whether Manifold has too much of a tech/AI focus to see if any users felt alienated by this part of the site. It will be interesting to see how strong the connection between Manifold, the rationalist movement, and tech really is.
If you want to learn everything that’s on the survey, well, you can just take it, but here are some of the parts that have gotten the most attention so far:
2. a. Manifold’s favorites
This was the section I was most worried about when I first created it. I made fourteen markets asking what Manifold’s favorite work was from different categories: movies, books, TV shows, songs, print comics, webcomics, video games, plays, short stories, board games, role-playing games, podcasts, YouTube channels, and poems. They all followed the same rules: Traders could submit any work in that category as an option, and I would include it on the survey asking users to rate their opinion of it: Excellent, Good, Okay, Bad, or Haven’t read/watched/etc. I then convert this into a numerical scale for range voting: Excellent = 2, Good = 1, Okay = 0, Bad = -1, and Haven’t read/watched/etc. counts as not voting, which is equivalent to adding 0 to the score. Whichever work has the highest total score will be deemed “Manifold’s favorite” in that category.
I was initially unsure of how this would turn out because to truly find out, say, Manifold’s favorite movie, I would need to get a lot of suggestions on what movies to include and a lot of people responding to the survey. It wouldn’t be a very good poll if people only submitted five movies and I just said, “Well, whichever of these movies Manifold likes best must be its favorite!” Luckily, I didn’t have that problem at all. Some categories got a massive number of responses, and even the more obscure categories that not as many people were into got enough to justify having a separate section for them. In fact, if anything I had the opposite problem: So many options were submitted that I was afraid people would get annoyed at how long the Manifold’s favorites section is. So far, people seem to like it though. I guess it’s not as long as it looks when you can just click “haven’t read/watched/etc.” for most of the options.
I’m pretty excited to see what Manifold’s favorites end up being, as well as what Manifold’s favorite overall work is. It will also be interesting to see which works are the most polarizing. I’ll probably make a market about that, too. At the time I’m writing this, the markets consider these to be the most likely choices for Manifold’s favorites:
Favorite movie: The Dark Knight
Favorite book: The Lord of the Rings
Favorite TV show: Breaking Bad
Favorite video game: Pro Evolution Soccer
Favorite print comic: Calvin & Hobbes
Favorite webcomic: xkcd
Favorite short story: The Last Question
Favorite play: Sweeny Todd
Favorite song: In the End (Linkin Park)
Favorite podcast: The Market Manipulation Podcast
Favorite role-playing game: Dungeons & Dragons
Favorite board game: Chess
Favorite YouTube channel: Destiny
Favorite poem: The Road Not Taken
Some of these categories are pretty volatile, though. For example, The Matrix and The Hitchhiker’s Guide to the Galaxy were ahead in the movie/book categories for most of the time the markets were open, they just happen to be behind right now. A few have been pretty steady favorites, though, mainly xkcd, The Market Manipulation Podcast, Chess, and Destiny. Though even out of these, the only ones that currently have a >50% probability for a single option are xkcd and chess. In other words, most of these are expected to be close races!
As a side note, I’m not sure what it says about Crystal Ballin’ that people consistently think Market Manipulation is more likely to win. Maybe not enough people have listened to it yet, or they need to get a second episode out before people rank it highly.
In addition to Manifold’s favorites in each category, I also want to find out which works Manifold judges as best, that is, which works are most popular among the people who have actually seen/read/etc. them? I made a market based on that premise:
I’m afraid that this one might end up with a lot of categories having different results, but not in a way that actually reflects Manifolders’ judgement. That’s because some of the works submitted were pretty niche, so there could easily be only one or a few respondents who are familiar with some of them, so we might get cases where the average is artificially high due to a small sample size. I made a second version to try to rectify that scenario:
I guess I didn’t help too much with that by submitting a few last-minute responses to these questions, some of which were obscure works that I like but might end up with very few votes. Although, not all the last-minute works I submitted fall into this category. There was at least one that I submitted just to find out what the people who have seen it think, because everything I’ve heard about it makes it sound bad, and yet it’s well-regarded.
There’s one more thing I should mention, which is the options that weren’t included. The only necessary criteria for an option to be included on the survey were that it actually existed, was a member of the category for the market it was submitted on, and wasn’t a duplicate of another option. I also stated that I could choose not to include an option under exceptional circumstances, but that this was very unlikely. The main thing I was thinking of here was if somebody submitted some horribly offensive work that I wouldn’t want to be associated with even through putting it on a survey, but I left it vague in case some other weird situation came up. Luckily, I never had to invoke that part of the description, but I did end up excluding some options on the grounds that they didn’t exist or weren’t a member of the right category. Two of these were pretty unremarkable: I didn’t include The Rationalussy Movie because there is no such movie (at least not yet), and I didn’t include “Post(@PosyMusic)” for the YouTube channel poll because that option was a typo, and the user who submitted it immediately submitted a corrected option after noticing the mistake. This wasn’t really necessary, since I would have accepted an option with a typo in it as long as I knew what it actually referred to, but I guess it’s an extra 25 mana of liquidity for the market. But there was one option that was controversial: Diary of a Wimpy Kid was submitted as a print comic on the last day before I published the survey, so I had to either make a quick decision about whether to include it or delay the survey coming out. The problem with Wimpy Kid is that it’s not a comic, but it does use some comic conventions like word bubbles and is sometimes classified as a graphic novel. In the description of the “print comic” market, I said that I would include graphic novels, but that was because normally, I think of a graphic novel as being a type of comic book. I have always considered Diary of a Wimpy Kid to be something else entirely, and I would have submitted it to the “book” category instead. At first, I thought I would include it out of a desire to be lax with what was included so that it wouldn’t seem like I was biased in my decisions, but after discussing it with some other users who all agreed that it wasn’t a comic or a graphic novel, I decided to leave it out. But if you disagree with my decision on that, there’s now a market where you can express your opinion:
If you want to see the markets that are specifically about the Manifold’s favorites questions on the survey, I made a group for them here: https://manifold.markets/group/manifolds-favorites
2. b. Is Manifold just a bunch of white guys?
Despite the topic of this market being something that a lot of people would just consider “boring demographic data”, it has managed to explode in popularity, probably thanks to the provocative title:
This has already become my most popular market ever (in terms of unique traders) by a pretty wide margin. The question it asks is whether at least 80% of the people who take my survey will say that they are white males (not including respondents who don’t answer the race or gender questions). Preliminary data (i.e., Pat’s Manifold Survey) suggests that this won’t be the case. He didn’t ask about race or ethnicity, but he asked the same gender question as I did and found that less than 80% were male. So I guess Manifold has a little more diversity than this market originally thought. This is nice for me as well, since it means I won’t have to parse the responses to see the overlap between white and male, I can hopefully just see <80% white or <80% male and say, “There, the proportion of white males is <80%.” I’m just so happy and relieved that I probably won’t have to do anything more complicated than that.
Well, shit.
2. c. What pill will Manifolders choose?
The second-biggest single market in my Manifold survey group is this one:
The market is based on this meme, which was the basis for Scott Alexander’s story, “…And I’ll Show You How Deep the Rabbit Hole Goes.”
(Sorry about the poor image quality, a high quality version doesn’t seem to exist.)
Although it’s quieted down a bit, it had a pretty lively discussion on which pill would be the most useful. As you can see, most people betting in the market settled on the black pill, though I still think the blue pill is better because you can use it in conjunction with relativistic effects to just do straight-up time travel, which is better than what the black pill gives you. Some people also like the orange pill, and there’s some debate over whether it should actually give you the abilities of all the other pills, since the abilities from those pills would be “activities that a human can do” in a world where they all existed.
Pretty much everyone agrees that the pink pill kind of sucks, though, given that it’s somewhat underwhelming compared to the others, has a Midas touch effect where you have to avoid touching someone to prevent the ability from turning off (at least according to Scott Alexander’s interpretation), and it’s just downright creepy. The traders also agreed that the green pill is a bad option, since it’s just so underpowered compared to the others. Although, the green pill might be more popular than the traders here thought, depending on the results of the next market…
2. d. How furry is Manifold?
As I already mentioned, Manifold has a lot of people who are into tech, and it’s a well-known fact that the probability of someone being a furry approaches one as their interest in tech increases. Some users on Manifold are open about being furries, and my last furry-related market got pretty good reception, so it’s probably safe to say that there are a decent number.
Of course, there is some ambiguity about what is actually required to count as a furry, and some people probably don’t self-identify purely due to stigma. So I made it a graded scale going from, “No, not at all,” to, “Yes, definitely.” As one user noticed, one of my options reproduces Moore’s paradox, the seeming absurdity of asserting, “It’s raining outside, but I don’t believe that it is.” Though in this case, the assertion is changed to, “I’m a furry, but I’m in denial about it.” I think it is a little more sensible to assert the latter, though. I can think of all sorts of situations of where someone would select that option, especially given that the survey is anonymous. For instance, you could be someone who Manifold has recently deemed “75% a furry”, and who Manifold users claim is a furry in denial.
Speaking of “Does Manifold think I’m a furry?”, shout out to Olivia for creating the market that inspired it. It ended up being pretty fun, we even had some intense forensic analysis going on.
2. e. Who should be Trustworthy.ish?
One question on the survey asked who Manifolders think should be Trustworthy.ish:
This is kind of like the favorites markets, except that it uses approval voting instead of range voting. According to the market,
has the most support, I have the second most, and @Stralor (i.e. Pat Scott), the creator of the original Manifold Survey, has the third most. We’ll see who ultimately comes out on top.This one is also one where I want to mention an option that’s not actually on the survey. In this case, it wasn’t because there was anything wrong with the option itself, but because it wasn’t submitted until after the survey was already released. @Joshua is definitely a good candidate for trustworthyishness, so much so that I think he might be inflating @JoshuaB’s level of support on my survey simply through people confusing the two of them (at least one respondent has done this).
2. f. Controversial market types
One of the questions was meant to gauge which types of markets are the most controversial. This was actually the suggestion I gave to the question on Pat’s Manifold survey asking what I would add if I ran a similar survey (I can now reveal this fact since my market on it has resolved). I included fifteen different types of markets on the survey, though there are probably some controversial types that I missed.
As you can see, most traders think that either whalebait or self-resolving will be the most controversial. I’m not surprised to see whalebait as the most likely option, but I’m surprised traders give self-resolving such a high probability. It seems like it should always be less controversial than whalebait, since whalebait markets are almost all self-resolving, and they’re (at least I thought) the most controversial kind of self-resolving market. However, traders have pointed out that some users might dislike self-resolving markets for reasons completely unrelated to whalebait, like the flood of “Will this market get X traders?” markets that happened after a few of them got big due to people encouraging new traders to bet to manipulate them. So I guess it’s plausible that people might find self-resolving markets annoying because of examples like that, but still have no problem with whalebait.
It’s also worth noting that the type currently given the smallest probability is poll-based markets. I guess that’s good since I recently made over 80 markets about the results of a certain survey.
2. g. The Possessive form of “Plasma Ballin’”
Ever since I changed my screen name to Plasma Ballin’, the question of how to turn that into a possessive has been looming over me. On the one hand, the standard way to do it would be to add ‘s to the end. That’s what Manifold does:
But that also seems really weird and unnatural. So maybe we should just ignore the rule that you add an apostrophe in this situation, since there’s already one there. That would result in Plasma Ballin’s. Of course, since this was one of the “fun” questions, I came up with some sillier answers as well, like the apostrophes cancelling out or combining into a quotation mark.
I regret not making this one where you can add answers, though, because as soon as I posted it, people started coming up with other funny ideas for what the possessive form could be. My favorite suggestion was from Lambda Fairy:
Mira gave a suggestion that’s extremely cursed but also makes perfect sense and is probably the grammatically correct way to do it in some contexts:
And @33cb gave whatever this cursed mess is:
Due to my regret over not adding these extra options, I did make the “Other” option a write-in, so that people who choose it can make up their own possessive. I am both excited and terrified to see what cursed answers you all come up with.
3. Easter Eggs
There are a few hidden surprises in the survey. In particular, there is a secret section to the survey that only some people will access.
I don’t want to give away anything about the section or how to access it, so I’ll move on quickly.
The other Easter egg is a linguistic illusion. Once again, I’m not going to give too much detail on that one yet to make sure it’s still a surprise. I made a market on whether people will be fooled by the illusion, although some traders don’t think my methodology for determining whether people have been fooled or not will be accurate. Keep in mind while betting that my method for determining this is based purely on people’s responses to the survey and no other data, and there may be some inherent issues causing me to over- or underestimate the number of people who were fooled.
4. Can I guess the responses?
After Pat published the results of his Manifold survey, I made a market on whether he would be able to guess which response was me.
He managed to guess correctly, and now it’s my turn to try to do the same thing.
This seems like it will be pretty hard - even though Pat guessed my response correctly, he admitted that he had no idea if he was right or not, and I don’t really have a good set of questions to use to single out his response. @evergreenemily also made a market on whether I could guess her response:
This one seems like it should be easier because I do have a set of questions that I can use to narrow it down: gender, furry, and Tears of the Kingdom (one of the video games in the Manifold’s favorites section). But we’ll see if those allow me to narrow it down to just one response, and if not, whether I’ll be able to guess correctly based on the other responses.
5. The survey image
I’ve been using the following image as a logo of sorts for the survey:
There’s not that much to say about it, but I figured I’d mention it here in case anyone is curious about where it came from. As you can probably guess, it’s AI generated, using leonardo.ai. I think the prompt was “plasma manifold”, but I can’t remember for sure.
6. What’s not on the survey?
6. a. Things I forgor💀
There were a few questions that I had intended to put on my survey but ended up forgetting about. I proofread the survey before release to try to catch as many mistakes as I could, but unfortunately proofreading doesn’t really work for this because, well…
But now that I’ve had time to think about it, I can remember some of the things I forgot. I was going to include a question about whether the respondents would be interested in participating in a Cursed Market Tournament, an idea I’ve been thinking about for a while but am reluctant to do because it would be pretty expensive, and its predecessor didn’t get much attention. Putting a question about it on the survey would have been a good way to gauge interest and to increase the attention. But, oh well, I guess the next best way to draw attention to it is to mention it here. I’m hoping to get at least a few people who commit to voting on each round of the Cursed Market Tournament so that we get enough votes to have meaningful results. If people are interested in doing that, then perhaps we can get a tournament going.
The other question I was going to ask but forgot is how we should determine which markets are non-predictive. This would have been a great thing to put on the survey because it is a pretty controversial issue on Manifold, and getting a large set of people’s opinions would be helpful to find out how people feel about it. I made a Manifold poll on it, but unfortunately, Manifold only supports “vote for one option”-type polls, which are a really bad way to determine people’s opinions on this particular question, unless I wanted to include an ungodly number of options. If it had been on my survey, I could have used checkboxes where users could select which things should count as non-predictive (i.e., one box for self-resolving, which presumably everyone would check, one for “any market where the market's existence could affect the outcome”, one for subjective, etc.). This is the thing I am most hitting myself for not including.
6. b. Questions that didn’t make the cut
There were some questions that I thought about asking but ultimately decided not to because I didn’t want the length to get even longer than it already was. Of course, I could have asked about more demographics and philosophical questions, but I cut it down to a few that I thought were the most interesting to see Manifold’s responses to.
If I were to re-run the survey, I would probably change the categories for the Manifold’s favorites section a bit. For example, when I asked about Manifold’s favorite RPG, I accepted video game RPGs as well as tabletop ones. I had originally been trying to keep the categories separate, but this meant there was overlap between the RPG and video game categories (and some games were submitted to both). So, maybe I should have made the market only about TTRPGs. On the other hand, D&D winning would probably have been a foregone conclusion in that case, and allowing video games led to a lot more engagement in that market, so maybe the way I did it was the right way to go.
There are also some categories of media that I could have included, but didn’t. Ultimately, the categories I used sort of became locked in after I started making meta-markets asking things like “Which category will the options with the highest score overall come from?” At one point, I thought of making one about Manifold’s favorite anime or manga, which I had originally overlooked because I don’t consume a lot of anime or manga. I decided against it on the grounds that they were already included in the TV show and comic categories, and therefore, there was no need for a separate category. But given that I ended up with overlapping categories anyway, that point seems kind of moot. I also could have split Manifold’s favorite movie into separate categories for animated and live action movies. That would have given animated movies a chance to win something, whereas it currently seems very likely that Manifold’s favorite movie is live action.
I thought about adding a question about the holidays people celebrate or their favorite holiday, but ultimately, that just didn’t seem as fun or interesting as the questions already on the survey, so I didn’t add it. I also came across this song and thought it would be funny to ask which method of obtaining cash from the song was the best, but I decided that with the survey long enough, I didn’t want to add a question that people had to watch a video just to answer. But oh well, God willing we’ll all meet again in Plasma’s Manifold Survey 2: The Search for More Money, and then I can include some of these cut questions.