Angry Monkeys

As part of our upcoming Karma system, we’ve been collecting a bit more information about who is doing your tests. One bit of information we’re collecting is referrals. Up until now if you bought a gold test and got 45 people doing your test, it wouldn’t matter whether those people were “random” testers or friends of yours that you had sent a link to. This meant that if you sent your link to 26 people you’d essentially be getting 26 less “random” tests.

Rather than discouraging this, we now ENCOURAGE it! If someone does your test having come from a link you sent out, we no longer deduct that from your “expected” results. What this means is that if you buy a gold test AND send your link out to 10 people, you can expect to get 55 results instead of 45!

By sending your link out, not only are you getting more results for your money, but you’re also helping other test owners get more results as well! Everyone wins!

Another minor update: We’re in the process of ditching “tokens”. They’ll still be used in the background, but we’re reverting to $ credit. Basically 1 token is worth $5. Everything will remain the same, it’s just that it will hopefully be less complicated than trying to work out how many tokens you need, how much the cost etc etc.

Comments(0)

We’ve had a couple of people contact us regarding the changes to Fivesecondtest.com. Not complaints, just suggestions. One user (Christopher) made a great response to my last post. I started to reply but then decided I’d just make a new post instead, as it is well worth the discussion!

You may or may not have noticed the last line of my original post, but it made reference to a “Karma” system. It is pretty much what Christopher described and it IS coming, we just haven’t released the exact details yet!

A couple of suggestions have been made, which I’d like to respond to here.

1. “Why not just set a cap on results to a certain number”
We can’t set “hard” caps on tests and responses. It may not be obvious, but the number of responses a test get varies with site traffic. Basically, the problem is that if we have a slow period and we have a hard cap we will end up with tests taking days and days to get all their results, or worse we may end up with no tests “waiting” in the system on a particularly busy day. We can’t afford to have people going to go do a random test and be told there are none to do!

In actual fact, when are traffic spikes it is often either just people DOING tests or just people CREATING tests and this makes coming up with a “cap” nigh on impossible. The number of “expected” results per test can vary from 9 responses all the way up to 20 or more (with premium tests being a multiple of that).

Now, what used to happen was that paid tests “expected” results were based on the average usage for the week. This means that variations to that number would occur slowly over time. When we saw a spike in traffic, rather than paid users reaping the benefits, free tests were “absorbing” that extra traffic (because when there are no paid tests awaiting results, we just grab the lowest free tests). We’d get free tests getting 20 or more results and bronze test (worked out on the weekly average) only getting 22 or 23.

What we’re now doing is making sure paid tests are absorbing that extra traffic, rather than free tests….hence why free tests are now fixed at 5 responses. When we get a spike in traffic, those extra results go to people have handed over their hard-earned cash. Of course, we still need to smooth the numbers a bit so that we have a reasonably consistent number from day to day, so to a degree free tests still absorb some of the traffic spike (at the moment most free tests are getting 7 or 8 results)….just not as much as they used to.

2. “Why don’t you reward people for doing tests?”
The karma system (which IS coming soon) will work pretty much how Christopher described it (i.e do X tests and get a free token). But it’s not as easy at it seems. At the moment tests have very little “noise” in their results. People are doing tests because they want to help someone else out. By relying on the good nature of people, we ensure that test results are good quality. People are filling out tests because they want to help, not because there is a reward for doing so.

The concern is that if we start giving away tokens for responses, we will be encouraging “noise” (i.e. people doing tests purely to get credits, not to be helpful). So what need to do (for memory tests) is allow test owners to vet their responses. They will be able to discard worthless responses and “reward” very useful responses. The result isn’t then just about how many tests you did, but also how useful your comments are. The problem here, however, is validating “useful” clicks in the click test. We are yet to come up with a good way to reward “good” clicks and deter “bad” clicks….how do you even tell the difference? That is the real stumbling block to the Karma system.

3. “can I just buy a subscription?”
We’d love for you to give us more money! But seriously, we’re not sure how a subscription results in better value for our users. Subscription models are great for the vendor, but I’m not convinced they’re better for the end user. The big upside for users, of course, is that they don’t have to deal with tokens (which we admit can be a little confusing!). The downside though is that having a certain number of tests available per work or month isn’t really how we’re seeing people work. I don’t ever want a situation where someone has bought 4 tests per month, and all of a sudden needs a 5th, but that requires upgrading to the next plan.

For me personally, the pay for what you use method is much better value for money. The vast majority of our sales are from people who buy tokens in bulk, and use them when they need them. If you need more, buy some more. We’ve just now included larger token packs to make this a whole lot easier to do this. Of course, if people keep nagging for subscriptions…we will add them! So keep your feedback coming!

4. Last but not least, “can I just see English/French/Chinese/German tests”.
Another tricksie problem! There are two ways to look at this problem. Test owners don’t gain much value from people doing their test if they can’t read the writing in it, and similarly users can get frustrated doing tests in other languages. The trick here is marrying the two together. We want to give the test owner the ability to exclude anyone who doesn’t “accept” that language, but we also want end users to be able skip all tests that don’t match their language choice. Doesn’t sound hard does it?

The problem arises in that most of our test “doers” are anonymous. Beyond checking their browser language we don’t know what they do or don’t understand. We also don’t want to trouble users while registering with too many demographic questions. So the solution at this point is to do a little of both. We’ll be adding the ability for registered users to submit their demographic information (if they choose!), and we’ll be adding the ability for test owners to exclude all but certain groups from doing their tests. We’ll also then have the ability for test owners to filter on that demographic information. We then just need to sort out the anonymous users….

Well, I hope that answers some of the questions floating around! We are still actively making changes, and so your feedback definitely DOES help us decide what to do next. We’re both pretty busy on other projects, so changes don’t happen quite as quickly as we’d like, but they ARE coming.

Thanks again for all your feedback and support!

Comments(6)

Hi All!

We’ve just made a couple of minor changes in preparation for some larger changes coming up in a few weeks time.

The first, is a bit of good and bad. The good news is that paid tests will now get more results than they’ve been getting! Hooray! The bad news is that it is at the expense of free tests…..

Now, now…let me explain!

Previously we worked out who got how many results based on the number of tests being created, and the number of responses being made. We simply divide one by the other. Free tests are worth 1, a bronze test is worth 2, silver – 3 and gold – 4. Add up all the responses in the last week, and divide by the number of tests (including the weighted values for premium tests)…that’s the number of “expected results”. Simply put Bronze tests get double the results of free tests, silver gets three times and gold gets four times. Easy.

With the increase in traffic to the site in the past few months, we’ve had a matching increase in responses to tests. Originally, we’d planned on the basic free test getting 5-6 results, the first level of premium tests getting about 12-15 results and so on. What has happened, however, is that free tests are now getting 12-15 results! In short, free users are reaping the rewards above our loyal paying customers.

So, it pains us to do it, but we’re capping free tests at 5 responses. We’ve spoken to a lot of people, and we feel that 5 responses is still enough to give a reasonable indication of a trend in testing. The upside of this is that the results that free users were soaking up will now be redirected out to paid users. Should you like more results at any stage, you can now upgrade an existing test to get more results.

Of course, as some of you may have noticed, we’ve been trialling a “bonus” upgrade to some free users. We’ll be ramping this up and upgrading more free tests (for registered users only) to “Gold” tests. Hopefully this will make up for those that feel they’re getting a raw deal! If you haven’t received one of these yet, hopefully it will be your turn soon. Keep in mind you have to be a registered user to be a winner!

The other change we’ve made is to pricing. Single tokens have gone up $1 to be a nice even $5. Buying in bulk still gives you a discount, but we’ve now added a 25 and 50 token pack due to popular demand. The cheapest per token price is still $2.50, and this is available when buying 50 tokens. We hope that the increase in test results you will now be receiving makes up for this small increase.

As always, we value your feedback, and we appreciate your support! Stay tuned for more changes coming soon, including the long overdue Karma system.

Comments(1)