A beginner’s overview of DevOps
Web Dev is hard
When learning web development, there is a lot going on, so much so that it is easy to get lost in the sea of moving pieces to the puzzle. One antidote to such chaos is to contextualize each new technology into a cohesive framework at a high level so as to build upon previous knowledge and add to your understanding. By stepping up to 50,000 feet and understanding the ‘machine’ as a whole, ancillary aspects of putting a new website or application out into the world become much more digestible. The aim of this article is to do exactly that, first delving into the role DevOps plays in modern web dev and second, outlining the key aspects of modern DevOps as areas worthy of further inquiry. Let’s get started…
Software is weird
Modern websites and applications have unique characteristics from a business perspective. They are one of the only places where mistakes are simultaneously unforgivable and inevitable. Users expect zero downtime and robust functionality to the point of perceived perfection, while developers and business owners necessarily make mistakes daily, if not hourly. The point of DevOps as a practice is to balance this mismatch of expectations and reality and bridge the gap between fallible systems and an enjoyable user experience. To be sure, there are three key aims of successful DevOps…
Reliability, scalability and iteration
DevOps is the equivalent of preparing to send your child off to college. It’s the thing that takes the proof of concept that works fine on your development machine, and gives it legs to go off and do great things in the hands of users everywhere. Indeed, when making the transition from using software locally on your laptop to providing it as a service to the world, one of the first challenges we face is making sure that it is reliable. This means that no matter what, our services work 24/7/365 with no exceptions. Users quickly lose faith in software even after just one unexpected error making uptime non-negotiable as a pillar of a successful software business (cite research). In addition, there is the problem of scalability. Having your software work for five people is a very different proposition from making it work reliably for 5 million people. The sheer volume of requests and computing resources needed complicates matter considerably. Furthermore, any great software product or website needs to evolve with its users’ needs over time. As with any sort of complex system, iterating and deploying thousands of changes will necessarily introduce errors into the mix. The issue of seamless iteration becomes clear when you understand that tech companies are deploying dozens of times a day and that most production codebases are hundreds of thousands if not millions of lines of code, the certainty of introducing fatal errors into your applications and the necessity of a system for handling them becomes clear.
What is DevOps?
The term DevOps refers to the collection of systems and practices that combine software development (hence ‘Dev’) and IT operations (hance DevOps) for the purposes of shortening the systems development lifecycle and providing continuous integration and delivery. In essence, it makes sure that you can deliver on your value prop continuously and mitigates the probability of someone on your team introducing fatal bugs into production code.
While the implementation details of the following products are no doubt involved and time consuming, I think you will find that a broad overview of some choice areas will give you the ammunition you need to start taking on the task of bringing your DevOps practices into the 21st century.
While sometimes overlooked, the importance of choosing a cloud provider for your computing resources cannot be overstated. While maintaining servers and managing custom hardware is certainly what’s called for in some specialized situations, cloud providers will no doubt be your best bet in the vast majority of cases. There is simply no way for small to medium sized businesses to manage their hardware more effectively. The costs associated with constructing and maintaining the requisite infrastructure necessary to compete is absolutely astronomical. Learning to leverage the agility and scalability of cloud infrastructure is absolutely crucial to securely scaling your services.
Tools of choice: AWS, GCP
Managing dependencies is tough enough by itself. By moving your software from your local dev environment to the cloud, the added complexity is enough to make anybody’s head spin. Managing system compatibility quickly becomes intractable as you move to cloud infrastructure over on-premise hardware. The solution? Containerization. By wrapping your applications in ‘containers’ you can effectively remove a layer of complexity and assure yourself your app will run no matter the hardware (within reason of course). This is done by taking a ‘snapshot’ of your dev setup and wrapping it up with a pretty bow so you can port it anywhere you’d like. Simple, effective, and absolutely crucial to keeping wtf’s/minute at a minimum.
Tools of choice: Docker and DockerHub
Once your application is running in containers in the cloud, there comes a time when you will need to make sure your services are healthy and operating cohesively as a unit. Container orchestration software and ‘Infrastructure as code’ solutions enable you to effectively administrate running containers without the need to ssh into ‘hot’ services in production. Additionally, you will need to ensure that your containers restart autonomously when they fail, allowing your cloud native container infrastructure to re-generate by itself without your involvement. Having a robust container orchestration and administration setup is a must, especially in small teams where an experienced sysadmin hire is not yet an option.
Tools of choice: Kubernetes, AWS EKS, eksctl & Flux
We’ve established that one of the key functions of DevOps is to ensure that fatal errors do not make their way into production code. In order to do that, testing your code before transitioning it into production seamlessly is the order of the day. A comprehensive testing suite is a staple in the CI/CD pipeline and cannot be overlooked a s key piece of successful DevOps within an organization. TDD, or test driven development, is something worth looking into, the intricacies of which are beyond the scope of this article. Suffice to say, up your testing game as soon as possible!
Tools of choice: NightmareJs, Jest, Selenium, Mocha
CI/CD stands for continuous integration & continuous delivery. These tools serve to orchestrate the entire DevOps pipeline step by step in an autonomous fashion so that you can ‘push to master’ and have your system do the rest of the work. When your CI/CD pipeline is finally up and running, it really is a thing of beauty. Instead of manually executing every step in the process of testing your updated code, pushing to master, ‘sshing’ into a hot server, using linux screen or another command line utility to pull your updated repo and making the transition between live endpoints across numerous servers and orchestrating the whole thing yourself, it is all done by your army of integrated workers! Your CI/CD pipeline will save you immeasurable time and headache, get it handled.
Tools of choice: CircleCI, Travis, Github Actions, Semaphore
While the magic and tangible benefits of a sophisticated DevOps pipeline are clear, there are most certainly tradeoffs. For one, understanding and implementing each piece of modern DevOps takes time. While the upfront investment is well worth it in the long run, the decision to implement it depends entirely on the vision you have for your project and whether the scale of the application requires the level of robust organizational systems engineering that enterprise grade software necessitates. Making that decision is up to you, it is a matter of right tool right job. If you are building a billion dollar company with a vision of having teams upon teams of engineers working on a multitude of codebases, then DevOps is an absolute necessity. If you are making proof of concepts for side projects, it’s really just a nice to have. Be judicious with your time! I will say however, that once you set it up once and experience the peace of mind and simplicity associated with a well-oiled DevOps machine, there is no going back!