startups

I’m currently in Sunriver, Oregon for the in-person component of the Neo accelerator. We’ve been here for almost two weeks and it’s been really nice: top quality food prepared for every meal, large and fully furnished houses, beautiful nature all around, talks/office hours with some truly legendary Silicon Valley individuals (Scott Forstall, Reid Hoffman, Alfred Lin), 24/7 access to our mentors, and most importantly, the ability to work side by side with the other startups in our cohort. Truly an ideal hack situation.

That being said, startups are really hard. I’ve read a lot about startups over the years, and people always said that “startups are always a lot harder than people imagine.” I didn’t fully acknowledge it, perhaps out of some misplaced ego. But yes, startups are a lot harder than I imagined. The hours are long, and being stuck in pivot hell can be frustrating and demoralizing. It is a strange feeling to work so hard/efficiently for 12+ hours a day for weeks straight without making substantial progress. We as a team (our startup is named Village, so we have been dubbed the “Villagers”) hit a new low point morale-wise yesterday. We were supposed to meet Erik Goldman for office hours for Product Design feedback, and our hope had been to have a product idea ready by said office hours (rather logical). However, in the hour leading up to the OH, we decided that our three day research sprint had failed to validate any of our product hypotheses. We were no closer than we were before the sprint, and our office hours mostly consisted of us explaining our current situation. Erik listened, explained that he understood exactly how we felt, and advised us to “go for a run.” We walked instead.

Despite all the hard points, I’ve been enjoying the process. I’ve never learned so rapidly before in my life–from figuring out how to do research effectively to learning more about DevOps than any human should ethically have to learn, the entire process has been really rewarding. It’s inspiring to be around smart people who are passionate about grinding on problems of consequence, and there’s a sense of comfort to know I can rely on my teammates. I really like everyone in the accelerator cohort, and the thought of being in the startup ecosystem/community for my foreseeable future is quite exciting.

sunriver

devops hypotheses

Our startup does not have a specific product idea, but we are generally interested in the DevOps space. I knew next to nothing about DevOps approximately a week and a half ago, but hopefully this means that I can explain it a little better since I also had to learn it from scratch.

DevOps’ official definition is “a set of practices, tools, and a cultural philosophy that automate and integrate the processes between software development and IT teams.” Essentially what this means is that developers want to push out code/product as fast as possible, but getting that code into a usable product that users/people use is its own process. Especially as the product scales/becomes more complex and more engineers join, reliability and workflows become especially important (this is part of operations). In the past, you’d have a separate operations team that would essentially oppose developers and ensure a set of guidelines for developers to follow to ensure quality production code/workflows. Over time, people realized it made more sense just for the developers to develop/enforce their own code quality guidelines and workflows, so that developers and operations weren’t silo’ed into opposing teams. Thus, DevOps as a concept was born.

A specific example of DevOps would be Continuous Integration/Continuous Development (CI/CD) and Testing. When a developer finishes a code change, they can publish said code. If the newly edited code base passes some set of automated tests, it is automatically integrated into the code that users interact with (continuous integration). Once it’s integrated, the newly edited product is published out into public for use in the real world (continuous development). This sort of workflow makes it such that the developer can focus strictly on writing the code, and everything else just happens automatically. Further, since every function runs in its own isolated code, systems built this way are much more flexible/amenable to change as you don’t have to worry about updates to sections of the code affecting other sections/engineers.

Our initial motivation is that startups rarely win on DevOps, but they often die on it. This makes it an interesting problem space to inspect, as if you can remove the hassle of DevOps from a company, they can focus on what actually differentiates their startup (product, business logic, etc.). Engineers are extremely expensive (typically 200k/year at least), tech startups have to move extremely fast, and they are willing to buy solutions rather than build it in house (cash is cheap, time is extremely expensive). DevOps is also extremely hard, and poorly planned out startups run into massive and often irreversible infrastructure issues once they scale (starts at around 30-40 engineers from what we’ve seen, and is a huge concern at 100 engineers). While most living tech companies have figured out DevOps solutions, there’s a whole host of hidden dead startups/ones that have never been started that have not. Because of this, we have conviction that eliminating the necessity for DevOps would unleash a new wave of startups/innovation.

serverless

The paradigm I outlined above makes a lot of sense when combined in conjunction with serverless computing. The term “serverless” is a little confusing because the server hardware/processes still very much exist. The chief difference of “serverless” apps is that they do not look at/manage those hardware/processes. To put this in historical context, tech companies used to have to spin up their own metal server hardware in order to host their applications. The invention of the cloud (i.e. AWS) made it so that you didn’t have to build your own physical servers–you could just pay Amazon a fee to use servers that they had already built out/distributed at scale. Virtual machines, containers (i.e. Docker) were all logical abstractions that closely followed. To me, the cloud feels like an innovation on top of the preceding status quo (physical servers). It was revolutionary in the sense that now startups didn’t have to spend valuable time/resources on building their own metal servers. However, anyone who’s dealt with DevOps/deploying on the Cloud/wrangling with Kubernetes knows just how hard/painful it is. Companies often need teams of DevOps or Amazon Web Services (AWS)/Google Cloud experts just to use the Cloud. Thus, serverless seems like the logical next generation of cloud computing, one where you don’t have to worry about server logic at all, saving precious engineer time (which comes out to be extremely expensive, even in just a monetary sense).

Our initial foray into serverless was driven by an interest in a newer area of Serverless dubbed “Functions as a Service.” I will refer to it as “FaaS” for the rest of this article. Fundamentally, FaaS is about running code without managing your own server systems. FaaS is interesting to think about because it enables the possibility of writing applications where server-side logic is still written by the application developer, but said logic is run in these mini, extremely efficient, compute containers. They’re stateless, event-triggered (say somebody clicks something on a website, then a Function will run as a direct response), and ephemeral (last for only that one invocation, which makes for extremely efficient compute pricing relative to having to keep a server up and running at all times even when nobody is using it). The best thing is that the underlying resource provisioning/allocation is fully managed by a third party–you can truly just write code logic without worrying how it manifests/reaches people in the real world. How you manage your infrastructure/servers hardly matters to virtually every company–especially early on, all you care about is the things that make your product different/good. How it gets to your user matters very little as long as at the end of the day it does.

This sounds great in theory, and the realization that many companies are still not built on serverless architecture was extremely exciting to us. Visions of a new generation of developers writing code and deploying products to the cloud with ease filled our minds/conversations. This was furthered when we realize that a big reason why serverless functions aren’t used more is because of cold starts–every time you start/shut down a function, it takes a significant amount of time. Thus, for any functionality that’s even moderately complex/used often, it’s probably better just to keep your server warm/running. This seemed like a hard problem, but not an impossible one–in fact, within a week, we’d managed to utilize WebAssembly technology to get cold start times for large Javascript functions down from 3 seconds to around 40 milliseconds (75x improvement!). This was far from being optimized as WebAssembly is still considered bleeding edge technology–there’s still quite a ways for it to grow, but the initial results felt promising. When talking to Reid Hoffman, we discussed our grander vision of building the next-gen cloud, seizing the cloud industry from AWS/Google Cloud/Microsoft Azure. Given that the global cloud market is projected to pass a trillion dollars in the near future, even we know that this was more of a faraway vision rather than a concrete plan, but you get the jist of the extent of the promise we saw.

Remembering that we shouldn’t get wrapped up in the tech as young founders in our position often do, we decided to take a step back and start doing deeper dives into DevOps.

entry points

We had this general conviction/hypothesis about serverless from our readings/intuition, but there was still a lot of work to do before we felt confident enough to start building a product (one of which is to come up with a product).

We started by interviewing the other startups in our cohort, which, by the nature of the accelerator/VC (Neo), all had technical cofounders. We identified specific pain points in the process of setting up infrastructure, and their solutions/workarounds. Every developer does things in their own way, and there was consensus from virtually everyone that dealing with DevOps was not something they wanted to deal with, especially at this stage of startups. People primarily focused on iterating quickly, they’re willing to buy rather than build, and everyone had small unique problems that ate up chunks of their time that they wished they could use for something more important. These points validated our beliefs regarding DevOps with startups.

We can’t exactly just push a serverless product on people and expect them to use it because we have some ideal located in our minds. During these interviews, we were looking for common problems that we could solve as entry points.

This is where we kinda got stuck. We were initially thinking about a Platform as a Service (PaaS)-like offering with additional auto-scaling/observability baked in, where you just hook up your code to our service and we handle all your infrastructure for you. It sounds like a great idea–what’s not great is that there are countless such offerings, some of which are billion dollar companies with infinitely more engineers than us. AWS Lambda functions seem like a really popular choice, and a Vercel/Next.js combination seems to be good enough from a developer experience that we can’t confidently say we could improve upon. We had found an excellent startup idea with billions of dollars worth of upside, but unfortunately, others had found the same thing, but a few years earlier. We could go head to head with these companies, but it felt more like a coin flip on whether or not we could win. When it comes to a startup that I’m realistically spending the next X years of my life on, it has to be better than a coinflip. Thus, we went back to our research/conducting customer interviews.

microservices

We had been focused on early stage startup developers as our customer segment the first week of the accelerator, but during our second week, we agreed that their needs were relatively well covered. Other problems typically are minor and summed up prove to be a problem, but it didn’t feel like we had an entry point for this customer segment. We decided to move up the stack and start looking at Series A and onwards startups (typically 100mil+ valuations).

There’s an interesting shift that occurs at this level because now you run into the issue of scale (number of users) as well as the complexity of code/engineers. When you’re an early stage startup, often you have less than 5 people working on the code base. Once you have multiple teams working on completely different parts of the code, new people coming in and experienced developers leaving, it’s a completely different kind of system level complexity. Early stage startups care a lot about new-idea-to-initial-deployment capabilities, but startups at later stages need to efficiently scale/build better features.

A lot of tech startups use a “microservice” architecture when they reach scale. This is in contrast to your standard “monolithic” application. Like what their names suggest, the traditional way of developing an application is to just have it all as one giant application (hence monolith). This makes it rather tricky to work on when it gets really big/complex–if different engineering teams are working on completely different things and yet their code is inherently linked, they can and do get in each other’s ways. A microservice architectural style instead develops the single app as a suite of small services, each running in its own processes and communicating with each other. This sort of modularity has many benefits, both from an organizational and technological perspective as each service are independently deployable and scalable. This sort of rigid boundary enable your teams to work indepedently, retaining the lean efficiency that is often lost as startups scale into larger startups.

Despite all the immediate benefits that come with a microservice architecture, after interviewing CTOs at many of these types of companies, it appears that having a centralized authority orchestrate all these different microservices is extremely difficult. You must balance the granularity of microservices (the smaller it is, the more efficient your application is) with the complexity of your overall system (the more microservices you have, the more integrations/connections you have). There are very real human DevOps concerns that come into play here, and most of the fires (metaphorical) that result are due to the company’s inevitable inability to efficiently manage immense complexity. Fires prove to be extremely costly, as they suck up all your engineering hours-if you’re perpetually stuck firefighting, you don’t actually have time to expand your business.

To that end, we explored the idea of having a graphical interface for DevOps. A logical interface for application architecture is a graphical one–people don’t interact with real-world architecture through command line interface and code. They use blueprints and physical models. There are a number of players in this space, and the idea of having a real-time, Figma-like, low-code GUI for DevOps seemed extremely cool at the very least. We put our heads down and dove into every possible vertical in security, testing, deployment, observability, and editing for three days straight (typing out every hypothesis that we explored would take far too long). After three days of countless interviews, articles, and trials of dozens and dozens of DevOps offerings, we had still yet to find a strong entry point to tackle with even this focused idea. This was when we entered our office hours with Erik demoralized.

I think we’ve just been overwhelmed by the sheer amount of information, so a main purpose of this article was to sort of step back and put together a general view of our progress/ideas. Everyone’s doing their own thing right now to take a step back, distill all the information we’ve collected over the past week and a half, and see if anything new pops out at us. I think the others are waiting for me to wrap up now, so I’m going to wrap up here/regroup to hear their thoughts and hopefully we can continue chugging forward. :)