Are Yahoo Mail users better customers than Gmail users?

September 28th, 2013 Comments off

I admit it: I judge you by your email address. Whenever I see a Hotmail, AOL, or Yahoo address, I can’t help but think that somebody didn’t get the memo in the mid-aughts about moving to Gmail.

It’s ridiculous. Somebody is no better or worse of a person for using Hotmail. All a non-Gmail, non-custom, address probably means is that the person is outside of my peer group. And yet the bias remains.

But what if that bias is justified? What if I can tell how good a customer will be based on the customer’s email address?

I run a µISV, in which I develop and sell blur reduction software called Blurity. As such, I have a large collection of purchase data, and that purchase data contains email addresses.

Let’s start with a look at the distribution of email address domain names. I queried the purchase logs for Blurity from October 2012 to find the percentage of purchases associated with each second-level domain name. Domain names that appeared in less than 1% of purchases were consolidated in an “other” group. From that, we can see that Gmail, Yahoo Mail, and Hotmail are the most popular email services for Blurity customers:

Proportions of email address domains for Blurity customers. Domain names representing less than 1% of customers are grouped in “Other.”

Not surprisingly, people also liked holding on to email addresses more commonly associated with ISPs, such as AOL, Comcast, Verizon, BT, and COX. (Time Warner fell just below the 1% threshold and is part of “other.”)

It’s tempting to say that Yahoo Mail is under-represented among customers, since in terms of US email service market share Yahoo Mail is roughly equal with Gmail. However, the share percentages are about equal for international users of the two services, and I couldn’t split my purchase data on international lines. Dead-end there.

So what can we look at? Maybe something to do with money? Well, how about return rates?

Blurity has a somewhat high return rate. Embarrassingly high. I attribute that to a combination of my poor UI design skills and the general misalignment between public perception of blur removal and the reality of blur removal. That, in turn, is complicated by the fact that many people don’t read directions, which is further compounded by people believing that the purchased version will somehow do better blur removal than the free trial version. As a result, I occasionally get emails asking for refunds. It’s the cost of doing business. I’d rather have a happy former customer than an angry customer.

I’d always felt that refund requests were more likely to come from the group of people using email services associated with lower degrees of technical prowess, such as Hotmail and AOL, so I was a bit surprised when I pulled the actual data. Turned out that the refund rate of Gmail users was not significantly different from that of Hotmail users, nor for Gmail versus AOL users.

Yahoo was a different story. The refund rate for Yahoo Mail users was higher than that for Gmail users at a statistically significant level (chi-squared, p<0.05). Found one!

That's enough data to recover the underlying numbers. Or is it?

Return rate by email domain name.

A higher return rate is nothing to celebrate, but it’s always nice to discover data vindicating one’s intuition.

Overall, Gmail was pretty good, or at least average. The only group that had a non-zero return rate significantly lower than Gmail was the “other” bin (chi-sqaured, p<0.01). The rest of the domains, including Hotmail, were all roughly the same, ignoring a few small-sample-size zero counts.

After seeing those results, I wondered what other predictors for refunds I could find in the data. I suspected that customers paying with PayPal might have higher return rates than those paying with Stripe. Maybe it was a manifestation of the general ill-will towards PayPal in the tech community? Much to my surprise, the refund rate for PayPal is actually lower than for Stripe (6.4% vs. 9.5%), though not significantly so.

I’m not sure that this is very actionable. The vast majority of customers of all sorts, including those with Yahoo email addresses, have been fantastic. I might try an A/B test for users entering Yahoo email addresses, perhaps with a more explicit link to the user manual or a tutorial video. Of course, if I think that might work for Yahoo users, then I might as well try it with everybody. Yahoo might have more refund seekers than other services, but it by no means has a monopoly.

Update: Had another customer make a purchase today and then almost immediately (three minutes later) ask for a refund. The customer’s email service? You guessed it: Yahoo.

Update 2: My friend Luke pointed out that the Live.com, Hotmail, and MSN domains are really all just Microsoft services. I agree, they should probably be grouped. I ran the numbers again with that grouping and found that the refund rate for the Microsoft properties does not differ significantly from that of Gmail, and the confidence interval is very similar to that charted for Hotmail alone. Thus, the conclusions above remain unchanged. Regardless, a good catch!

Pulp Entrepreneurship

September 16th, 2013 2 comments

With apologies to Quentin Tarantino.

At a table inside the Coupa Cafe, a coffee shop in Palo Alto,
California.  VINCENT and JULES are eating breakfasts of
eggs/sausage/pancakes and a muffin, respectively, and drinking
coffee.  They are talking about entrepreneurship.

                JULES
        Yeah, I've just been sitting here thinking.

                VINCENT
        About what?

                JULES
        About the startup I've got cooking.

                VINCENT
        Startup in your mind.  I think you don't have anything
        more than a weekend hack.  Maybe a web app at best.

                JULES
        What is a startup, Vincent?

                VINCENT
        A company that's just getting going.

                JULES
        And what kind of company would qualify?

Vincent takes a sip of coffee

                VINCENT
        It's... a company that has aspirations of growth, of making
        money, 	of changing things for the better.

Jules points at Vincent to indicate that he's hit on the main idea.
Vincent pauses, and then continues.

                VINCENT
        But your little side project, I don't think it qualifies.

Jules CHUCKLES.

                JULES:
        Hey Vincent, can't you see that shit don't matter?  You're
        judging this shit the wrong way.  I mean, it could be that
        my company will hit it big, or that Snapchat won't just be a
        fad, or that some 16-year-old will flip a stupid vampire
        social network to some dumb 18-year-old wannabe hedge fund
        manager.  You don't judge shit like this based on merit. Now
        whether or not what I'm doing is an according-to-Graham
        "startup" is insignificant.  But what is significant is that
        I feel it in here. 

Jules points to his heart.

                JULES:
        I know that what I'm doing is a startup.  Call it what you
        want, but it's a startup to me.

                VINCENT
        But why?

                JULES
        Well, that's what's fucking with me.  I don't know why.  But
        I know it's what I'm meant to do.

Vincent SCOFFS.

                VINCENT
        You're serious?  You're really thinking about quitting?

                JULES
        Software consulting?  

                VINCENT
        Yeah.

                JULES
        Most definitely.

                VINCENT
        Fuck. You're making such great money!

Vincent SIGHS.

                VINCENT
        So what are you going to do then?

                JULES
        Well, first I'm going to deliver the remaining code to my
        client.  Then, basically I'm just going to do the startup thing.

                VINCENT
        What do you mean, "do the startup thing?"

                JULES
        You know, like Zuckerberg and Facebook. Raise money, write code,
        change the world.  

                VINCENT
        And how long do you intend to "raise money and write code"?

                JULES
        Until my startup takes off and I have a successful exit.

                VINCENT
        And what if you never have an exit?

                JULES
        We're in a bubble.  Somebody will buy it.

                VINCENT
        So you decided to be an asshole. 

                JULES
        I'll just be Jules, Vincent.  That, and I'll be incredibly
        rich eventually.

                VINCENT
        No Jules, you decided to be an asshole.  Just like all of
        those pieces of shit writing pretentious blog posts and
        going on about social-this or mobile-that.  Who pretend
        their MBAs qualify them for eight-figure VC investments,
        or who treat content farming like it's adding utility to the
        web.  They have a name for that, Jules: an asshole.  And
        without a revenue plan or a product that will scale that's 
        all you're going to be at your so-called "startup": a 
        fucking asshole.

                JULES
        Look, my friend, this is just where you and I differ.

                VINCENT
        Jules, what happened over the past few days, I agree,
        some angels showed some interest and your stub of a signup
        page collected a few email addresses, but an actual
        startup?  I don't think...

                JULES
        Any idea as long as it can get traction, Vincent.

                VINCENT
        Don't fucking talk that way to me, man.  How will you make
        money?

                JULES
        If my answers frighten you, then you should cease asking
        scary questions.  

                VINCENT
        Let me ask you something; when did you make this decision?

                JULES
        Just recently.  I was sitting at home, reading about how
        Twitter was about to IPO, and how Uber picked up $250 million 
        from Google, and I had what techies refer to as, "a moment 
        of envy."

                VINCENT
        Fuck.

When robots flew

June 25th, 2013 Comments off

I was getting out of my car when a small plane buzzed over my head. Sam was parked in a grass field amongst many other Subarus and SUVs. The clouds above were breaking; it was going to be a hot day in Boulder.

I let my gaze follow the plane as it buzzed down the field. A wide gate was set in the air like a high-jumper’s bar, but instead of going over it, the plane went below it. In the distance, I could hear a roar of approval come up from the crowd of spectators. It would have been a trivial flight maneuver for a human, but no human was at the controls: the plane was flying autonomously.

When I think of robots, I think of autonomy. Sure, remote-controlled devices may technically qualify as robots, but that’s always seemed a bit like cheating to me. Imagine my thrill, then, when I found out that one of the largest competitions for amateur autonomous robot designers would be held an hour from my home in Denver.

The Autonomous Vehicle Competition draws participants from all over the country, and indeed, the world. It’s the effort of Sparkfun, a Boulder company that supplies electronic parts and kits to hobbyists of the robot kind. The competition is split into two different challenges: one requires navigating a ground course, roughly a square 100 ft on a side in a parking lot with obstacles; the other requires flying from a judging area over about 300 feet of water, crossing a peninsula, and returning to the origin. Each run is scored according to navigation time and extra challenges completed.

A ground-course vehicle leaps over the finish line

About a decade ago, my friend Joey and I made a GPS-guided self-driving Lego car for a class project. It was crude and didn’t work very well, but the thrill of having wrought into existence something that could actually drive itself around — well, I was all smiles even back then. The same enthusiasm was written on the faces of the hobbyists and engineers at the competition. I’m sure the same excitement is shared by those people working on the slightly bigger toys at Stanford and Google.

A large crowd of spectators packed the viewing stands and spilled out along the safety fences and grass. They were young and old, overwhelmingly male but not wholly without female representation. Some were entrants waiting for future heats; others were simply there for a passive thrill.

I spent some timing watching the aerial competition before moving to the ground track. Much to my surprise, the plane I had viewed upon my arrival was one of just a few fixed-wing entries. Though there was one true helicopter in the mix, the vast majority of aerial bots were quad- or octo-copters.

One of many quad-rotor entries; but unlike the others, a successful one

I’ll admit that a certain part of me wanted to see a big splash in the aerial competition, but that didn’t happen: the designs were surprisingly resilient, albeit not often successful. Also, the makers were generally standing by ready to cut over to manual control should everything go sideways.

Some entries ran the course and chalked up bonus points without breaking an electronic sweat; others failed to do much more than lift off the deck, hover for a few seconds, and set back down.

Over at the ground competition, the range of entries was a diverse affair. Everything from self-balancing two-wheeled contraptions the size of a coffee cup to a big-wheel tricycle made attempts on the course.

A “big wheel” is chased by a mini Segway

The course proved deceptively difficult. Very few of the ground vehicles successfully navigated the circuit. For at least half the entries, that meant failing to make the first of the four turns.

Some bots slammed into the far fence. Others turned too soon. A couple spun around at the starting line and zoomed off at high speed the wrong way around the course. The defending champion’s bot got loose in the infield, leading to a college-age nerdy guy running as fast as he could to chase it down, with the bot swerving to and fro as if it were trying desperately to escape some tyrannical master.

Spectators, spectators everywhere

Oh, and crashes! During the unlimited-class heat, I got to see the carnage I had paid (nothing) for: one smallish slammed itself into the far fence. Meanwhile, a large gas-powered go-kart bot gained speed off the starting line, started going for the first turn, and then apparently decided to give it a miss and accelerate into the fence — right where the small bot was hung up! The go-kart tried to get away and dragged the small bot a good 20 feet down the fence, leaving bent wire and miscellaneous bits of expensive plastic in its wake.

Crashes! A go-kart takes out a smaller bot, much to the amusement of the non-owners

During a lull in the competition, I took a walk through the team pit tent. At table after table, teams (and they were teams — I saw almost no solo entries) were hunched over laptops, sifting through telemetry, tweaking code, and mending broken bot bodies. The people were as dedicated as any I’ve seen in any other competition.

A college-age man works on his robot in the pit tent

And yet, the mood remained light. There was an overwhelming air of fun and excitement. Even the entrants whose bots had failed wore big smiles as they carried their creations off the course and developed stories of what went wrong and how the big victory got away. Fishing stories are not just for fishing.

Entries ranged from the simple (an 8-year-old boy made it half-way around the track with a Mindstorms-based four-wheeled bot) to the ultra-sophisticated (the university quad-copter teams running using GPS, computer vision, and other whiz-bang gadgetry). Complexity seemed to be poorly correlated to success, at least on the ground course: the slow and steady bots, apparently using dead-reckoning, generally performed more reliably than their fancier, faster, GPS-enabled, optical-sensor-encrusted cousins.

This young girl’s robot had no shortage of anthropomorphic flare

I felt more inspired than ever to make an entry for next year. How hard could it be?

 

My web browser’s view of me

April 7th, 2013 Comments off

What does your web browser think you like?

Many modern browsers, including Google Chrome, have a nifty predictive-text feature in the address bar.  You start typing, and it predicts what you’re trying to say. This works even after a single letter. The results are shaped to each individual based on one’s browsing and search histories.

So, I got to thinking: which web sites does Chrome predict for me for each letter of the alphabet?

Here’s what I found:

aamazon.com (I recently got an Amazon Prime subscription, and it’s pretty awesome)

bblurity.com

cclassic.wunderground.com (I like the classic interface much better than the “normal” one)

ddigg.com

eebay.com (A bit of a surprise, as I haven’t bought or sold anything on eBay in a while)

ffacebook.com

ggoaliestore.com

hhipmunk.com (Best travel search site at the moment)

iistockphoto.com (I sell some stock photos here. I also seem not to visit many other “i” sites)

jjoin.me (I must not visit many “j” sites either)

kkeacher.com

llinkedin.com

mmaps.google.com

nnews.ycombinator.com

ookcupid.com (Still searching for love!)

ppaypal.com (About 30% of Blurity purchases are via PayPal)

qqrz.com (Very surprised by this one — I think I’ve been here once in the past year, and the site isn’t even all that useful)

rreddit.com

sslashdot.org

ttheonion.com

uuspto.gov (Google is better for searching patents, and freepatentsonline.com is better for viewing them, but only here can I view the PAIR data)

vvail.com

wwellsfargo.com

xxkcd.com

yyoutube.com

zzappos.com (Per my records, the last time I bought something from Zappos was August 2007)

 

Based on this data, what picture can we draw of me?  Maybe that I’m a tech-savvy engineer who likes to travel, laugh, keep up with friends, and buy and sell things on the internet?

The problem with this data is that it is not directly correlated to the frequency of visits. For that, it’s more useful to look at Chrome’s “Most Visited” sites, visible when a new empty tab is created.

Thumbnails of the thumbnails of my frequently visited sites

For me, the sites are:

  • reddit.com
  • okcupid.com
  • facebook.com
  • news.ycombinator.com
  • cnn.com
  • blurity.com
  • craigslist.org
  • wellsfargo.com

Curiously, two of the entries on this list (CNN and Craigslist) were absent on the predictive list. This gets at a second problem with predictive input: it doesn’t work as well when there are multiple sites that start with the same letter or sequence of letters.

I wonder how self-reinforcing these sets of sites are. That is, am I more likely to continue to visit them because the effort required to change is higher than to not change?  It’s possible.

CNN in particular might be an example of that. Most of what they publish comes from the wires, so I could just as easily make something like the Denver Post or Reuters my hard-news mainstay. In fact, I’ve done exactly that on my phone. Still, the motivation to switch on my desktop computers is lacking.

It should be interesting to see how this changes over time. I expect that Hacker News, Wells Fargo, and Blurity will stay on the list for the foreseeable future. OkCupid may drop out if I manage to find that special guy. And Facebook? It’s an unknown. Will I still care about it in four years? Not sure. The next shiny thing might have come to the fore by then.

How we got a Y Combinator interview but blew it

October 24th, 2012 10 comments

Six months ago, I got a scratchy cell phone call from Tyler while he was hiking at Arches National Park:

“We’re going to Mountain View!” he exclaimed.

“What?” I asked, scared I had misheard him.

“Y Combinator said ‘yes’ for our interview!” Tyler clarified.

“Woohoo!”

A week later, we were in Mountain View.  We interviewed in the afternoon, and in the evening we were rejected.  Here’s how we got to that point.

Note that I have not titled this “How to get a Y Combinator interview” — that’s a different problem.  This post is simply about what we did.  If you want a proper guide, I suggest Jason Shen’s Unofficial Y Combinator Guidebook.

Y Combinator Funding Application Summer 2012

Company name:

Snaposit

It started one evening in early February 2012, when Tyler pitched an idea to me: “Backup for photographers.”

Since we both dabbled in photography, that seemed like a real problem, and after some cursory research, we felt we could solve it.

The gist was that photographers generate so much data that it’s impractical to do backups over consumer internet connections. We polled some of our photographer friends: Did they currently have an off-site backup solution? Would they pay to have that problem solved? Encouraged by the responses, we pushed forward with our solution: Snaposit.

What is your company going to make?

Offsite backup service for photographers

We decided that our first deadline for Snaposit would be the Techstars Boulder application. Both Tyler and I lived in Denver, so it was an obvious choice. The early application due date was  February 26th, so we filled out the TS application form, spent too long over-producing an application video (which was viewed precisely zero times by Techstars), and sent it in. Fairly quickly, we got a question back from Nicole Glaros of Techstars: when would a prototype be ready?

It was a good question, because it prompted us to actually build the prototype. In a mad dash, we put something together in just a few days. The result was horribly ugly and barely functional, but with the clock ticking, we sent it along anyway. Unfortunately, it was Windows-only, and Nicole seemed to have a Mac. Oops.

For each founder, please list [various biographical info]:

tghw; Tyler G. Hicks-Wright; 2007, Stanford University, MS, Computer Science (Artificial Intelligence); 2005, Rose-Hulman Institute of Technology, BS, Computer Science & Economics (Double Major); Homepage: tghw.com [portions redacted]

teuobk; Jeff Keacher; 2009, Stanford University, MS, Management Science & Engineering; 2004, Rose-Hulman Institute of Technology, BS, Electrical Engineering; Homepage: keacher.com [portions redacted]

Needless to say, we got rejected by Techstars. We laughed it off, called it nothing more than a practice run, and buckled down so that we’d have a really good prototype to show to Y Combinator.

Why did you pick this idea to work on? Do you have domain expertise in this area? How do you know people need what you’re making?

We are both avid, published photographers, with huge photo libraries that we have not been able to reliably back up off-site. We are friends with and have worked with other professional photographers who have expressed that they have the same problem.

As the summer Y Combinator application deadline loomed, we filled out that application and made another video.

Oh, the video. We spent a lot of time on the video, which we justified by thinking that a crummy video made by people claiming to know photography probably wouldn’t be very credible.  I’m not sure that it mattered in the end, but nobody could argue that our production values weren’t high.

Please tell us about an interesting project, preferably outside of class or work, that two or more of you created together. Include urls if possible.

How about an adventure we had together instead? Like when the two of us were backpacking in Denali National Park, Alaska, where we were nearly killed by hidden waterfalls and again the next day by grizzly bears. But both times we overcame!

In the software realm, we’ve successfully created the Snaposit beta together in under a month, with each of us developing a major component.

We set up good lighting (two 500-watt halogen lamps into two silver umbrellas, one on each side of the camera, plus a large white disc reflector on the table in front of us for fill). We shot with a good camera (Canon 7D in 720i mode). We used a dedicated microphone (Zoom H4n’s built-in mics, just out of frame above us).

We did take after take. We drank whiskey. We debated what to cover and what to leave out.

Finally, after a couple hours of takes, we did the final one: the one that would become the submitted video.  Although parts of the video might seem a bit scripted, it was all improvised. Honestly, we probably would have done more takes had the camera’s battery not died about 10 seconds after the final take concluded.

Was the video kind of cheesy?  Yes.  Were dark shirts a bad idea? Sure. Did it work? Apparently.

The video (posted on Posterous) was our only real tool to gauge interest in our application after submission.  We watched the view counter slowly tick up as the days went buy.  As the views went up, so did our excitement.

Our dream came true; Tyler relayed the good news to me.  We booked flights to San Francisco, celebrated, and buckled down to really refine our demo for the YC partners.

The morning of the interview found us in the upper floor of Red Rock Coffee Shop in Mountain View.  We were banging away on the code, trying to deal with some major performance bugs that had emerged when we switched to a local demo web server.

If you’ve already started working on it, how long have you been working and how many lines of code (if applicable) have you written?

We have been working on the code for about a month. Currently, we have about 3,000 lines of Python in the cross-platform client and another 2,500 lines of Python, HTML, Javascript, and CSS in the website.

About half an hour before our scheduled interview time, we made the short drive from downtown Mountain View to the Y Combinator offices.

I’d never before been somewhere so orange.  Hundreds of young guys — they were almost exclusively male, and most seemed to be in their 20s — milled about the large common area.  Laptops bloomed from tables like daffodils in the spring.

We checked in and went through our demo a few more times.  Several other people tried to chat us up, but we politely declined.  We were focused on the interview, and everything else before then would just be noise.

Finally, the time came.  And went.  And we still were outside of the interview room.

Paul Graham came out and made himself a smoothie.  He needed a short break, he said. He was barefoot.

Please tell us something surprising or amusing that one of you has discovered. (The answer need not be related to your project.)

* USGS topographical maps in Alaska are not detailed enough to show hidden waterfalls. Discovering this fact was nearly catastrophic for us.
* The best cinnamon roll yet discovered is about 75 miles north of Fort Nelson, BC
* eBay generates almost 27% more revenue on Sundays than on any other day.

Since research has shown that judges are more lenient on full stomachs, Tyler and I thought it was great that Paul was eating something.

The interview began.  Paul, along with Trevor Blackwell and Robert Morris, grilled us on our business model over and over.  They seemed to accept our problem as genuine, but they seemed unconvinced that our solution was the right one.  Minutes went by.  I felt that I wasn’t paying enough attention to Robert, but I couldn’t seem to fix it.

More time went by. They kept wanting to compare Snaposit to Dropbox, and we kept trying to tell them that we were solving a different problem. Finally, the 10-minute timer beeped zero — and we hadn’t even shown them our demo.

We raced through the demo of our prototype, and the three YC partners seemed unimpressed.

We were ushered out of the room, and I knew that we had failed to make a good impression.  We should have been better prepared to defend our solution.  At the same time, we shouldn’t have appeared so set on our existing solution and expressed more willingness to adapt (should it prove necessary).

Our heads were swimming as we left the YC building. We decided liquor was in order.

We spent a couple hours at a bar in downtown Mountain View and then met some friends from undergrad at a nearby restaurant.  All the while, we were waiting anxiously for a phone call from YC.  It never came.

Halfway through dinner, we got the rejection email.

Start of the YC rejection email

Tyler and I resolved to continue working on Snaposit, and so we did. The private beta led to the public beta, which gave way to the general launch. However, the YC partners’ concerns proved prescient: we had identified the right problem but the wrong solution. Not enough people wanted Snaposit.

After evaluating the opportunities for pivoting and surveying our non-customers to find out what would make them happy, we decided that there were no realistic paths to salvaging Snaposit. We tried, we failed, and we were going to move on. Snaposit will be dead by the end of the year.

In the meantime, my other photography software tool has taken off. Blurity, a tool for unblurring blurry photos, is doing great.

Tyler is also doing well, with no shortage of projects to keep him busy.

Snaposit didn’t work out, but we had a great experience building it. Interviewing with YC was fantastic, and the partners were even sharper than we gave them credit for. Was it all worth it?  Without hesitation: yes.

(For an alternative treatment of this story, see Tyler’s write-up from May and his later discussion about why Snaposit failed)