How Tony Dinh tried to improve conversion using an in-app survey

notion image
Tony Dihn, founder of Blackmagic.so — a tool to manage and grow Twitter audience — decided that he could try to improve the conversion rate by running a simple poll.
In May 2022 he asked people why didn't they complete the sign-in flow.
The answer was unexpected.
notion image
notion image
According to the survey, the majority of users that did not complete the sign up flow did it because they had concerns about permissions (they were probably thinking the tool would be posting someting on their behalf without them knowing it). Tony changed the onboarding flow. Blackmagic.so started to ask for Read-Only Twitter permissions when users signed up. When a user wanted to get access to more advanced features that need the “Write” permission, they had to grant it separetely.
Did it help? Nope!
Nothing changed in the sign-up numbers as well as in the conversion rate — if anything it went down.
notion image
Source: Tony Dihn monthly newsletter
Why it had an opposite effect? Because surveys are not the best way to get insights into customer’s behaviour. Sometimes people don’t have a relevant answer so they just choose the first option. Sometimes they don’t have time for the survey but because they like the founder they agree to spend 2 min unswering the questions — but again, they don’t take it seriously and choose the first option. Finally, people are really bad in understanding the tru reasons behind them doing something or not doing something. They don’t lie — they just don’t know the reason. Besides, how statistically significant can be the results of a survey like this? Who would agree to answer questions about an app they did not even sign up for? Probably, people who are NOT your targeted audience. Therefore, whatever they say, should be of no concern to you.
Tracking actual behaviour and talking to real customers is a more relevant way to craft a better onboarding.

Like what you’ve read so far in your founder guides? We’ve got more to come

Upcoming guides...

  • Content marketing distribution machine: a full guide for founders
  • Founder guides: create an affiliate program
  • A founder guide to programmatic SEO
Co-founders & Hiring
  • Founder guides on hiring your first employee
  • Outsource effectively
  • How to split up equity
  • Finding a co-founder
  • How to find relevant beta users
  • Prioritize features effectively
  • How to price your product
  • Building a roadmap
Have an idea for the next Wizen Guide or interested in contributing in the next founder guides? Give us a shout at [email protected], we’d love to hear from you.


Meet the team 👋

notion image
Ana Bibikova - @NotechAna
Ana is an Author, Marketing Strategist and a Mentor in the Founder Institute. Ana has 18y of experience in building businesses, growing them to $4M annual revenue.
notion image
Anthony Castrio - @AnthonyCastrio
Anthony is a Software Engineer, Fractional CTO, and the founder of Indie Worldwide, a virtual incubator and social club for bootstrapped founders.