The 6 Pillars of Software Engineering
As a Software Engineering Manager, a lot of people on my team had asked me: how can they become a Software Engineering Manager? And even some people have asked me: how can they become a CTO?
Becoming a CTO might be the highest level you can achieve as a Software Engineer. I mean, it might be higher levels than CTO like becoming a CEO or winning a Nobel prize for writing an ultra-compression algorithm and achieving world peace, but for now, let’s say you just want to become the Ultimate Software Engineer.
That makes me think of the UFC (Ultimate Fighting Championship), which is a company that promotes mixed martial arts, including but not limited to: Boxing, Kickboxing, Jiu-Jitsu, Sambo, Muay Thai, Karate, Taekwondo, and Judo.
For a fighter that only knows 1 martial art is almost impossible to win this kind of competency. The knowledge of multiple styles allows them to have a variety of options to choose from at the right moment to get the best results.
And I think that is exactly what it takes to be the Ultimate Software Engineer; to reach a good level of expertise in multiple disciplines. A person that can think on the best approach for any kind of Software necessity by choosing the correct components to use between a wide range of options.
What components? Well, is the year 2021 and at this moment we have 6 main pillars:
There are a lot of options in this area, let’s check the back-end languages by popularity and highest salaries worldwide:
Which one do I recommend? Well, I can’t tell. First of all, I don’t know you, I don’t know who you are, I don’t know your background. But, let’s assume the following backgrounds:
If you are a DevOps engineer with a particular cloud experience check this diagram of correlated technologies, it will give you an idea of which cloud services are related to which languages (i.e. Azure is highly related to C#):
If you don’t have any background I have heard that PHP is easy to learn. But, C#, Java, and Python are at the top 3 of popularity and have very good support, documentation, and StackOverflow answers for almost any problem you might face.
We see that jQuery is the most popular, but over the years is losing ground to React and Angular. So, I don’t recommend you learn JQuery because most companies are moving away from that framework.
React is owned by Facebook, and Angular is owned by Google. The good thing about React is that it has a Mobile version called React Native, which allows you to develop Android and iOS apps using the React Framework.
Being React more popular, and having the mobile version (which BTW also produces a web version; yes, 1 code, 3 assemblies: web, android, ios), my vote is for React.
In this area, we have 2 main types of modern databases to choose from: relational and non-relational, also known as SQL or NoSQL (for their query languages).
SQL databases are known as relational databases, and have a table-based data structure, with a strict, predefined schema required. NoSQL or non-relational databases can be document-based, graph databases, key-value pairs, or wide-column stores. NoSQL databases don’t require any predefined schema, allowing you to work more freely with “unstructured data.”
You don’t have to go just in one direction (either SQL or NoSQL), you can do both; remember that NoSQL doesn’t replace SQL, it complements it.. and vice-versa.
If you are uncertain about which kind of DB you should use, answering these questions can help you make the correct decision:
- What kind of data will be stored in the database?
Are you storing log files or user accounts?
- How complex is the data that will be stored?
Can the data be normalized easily?
- How uniform is the data?
Does your data roughly follow the same schema or is it disparate or heavily nested?
- How often will it need to be read or written?
Is your application read- or write-heavy, or both?
- Are there environmental or business considerations?
Do we have existing agreements with vendors? Do I need vendor support?
Now, let’s see what developers are most interested in:
Redis can function as a NoSQL database, a cache, and a message broker. Cache is the classic function of Redis. Essentially, Redis sits in front of a disk-based database and saves queries and results; the application checks the Redis cache for stored results first, and queries the disk-based database for results not currently in the cache. Given the sub-millisecond response rate of Redis, this is usually a big win for application performance.
Since Redis was the first option for Cache, most developers thought that it would be better to just use Redis as a database too, instead of integrating another database provider. And suddenly, it became very popular.
Anecdotally, Stack Overflow has been using both Redis and Elasticsearch in their tech stack for years, since the company's early days. MongoDB remains the database technology that developers want to learn the most.
The logical suggestion here is to learn Redis in the NoSQL area. And I also recommend that you learn a SQL Database; I can agree with the image above that PostgreSQL is a good option, so, I’ll start with those 2.
Dev-Ops is the merge of 2 words: Development & Operations. We cover Development in the previous topics. So, now let’s define which Operations we are talking about.
The Operations that development usually needs at the most basic level are code compilation and publishment. But you might also need other steps like setting up your environment particularly (if you have pre-production environments), code reviews/approvals, running tests, code analysis tools, dependencies accesses, etc.
There are a lot of options that offer you more than 200 services. The most popular (in this particular order) right now are:
I don’t have a preference for either one of these platforms. In this case, you should check the Correlated Technologies — How technologies are connected figure in the Back-End section of this article to see which make more sense for you to learn; i.e. if you are a .Net (C#) back-end developer it will be better (but not mandatory) to learn Azure.
Is good to mention that IBM has made good progress in the Quantum computing race by installing the first retail quantum computer in the US. Imagine if they somehow figure out how to use quantum computers for their servers and you can use their quantum power to host your systems.
On the other hand, Docker and Kubernetes work very efficiently with microservices, and a lot of the biggest companies in the market (Netflix has more than 200 microservices) had implemented that approach. That’s why Docker and Kubernetes are at the top of popularity. So, I strongly suggest you learn about them. All 4 of those clouds support them.
Dev-Ops is a large topic, each Cloud handles more than 200 services. One of those services that are very important to mention is the CI-CD pipelines. I recommend that you first learn about the CI-CD pipeline that is already integrated with the Cloud service you choose. And then is always good to consider other options like:
Computing power has been increasing exponentially through the years, and with great power comes great creativity. We started by replacing simple human manual tasks with software, then we automate processes to perform human repetitive tasks, and now we are teaching computers to learn, understand and make decisions, and take action by themself.
In the future, I guess our goal is to let computers do all the human work so we can be free to do whatever we want, without having to worry about human necessities. Computers will be our autonomous providers; a colony of ants with only one goal in “mind”: to supply humans with everything (food, home, love, entertainment, health, etc).
I hope that happens soon, so we can finally chill and play video games all day long.
And for that to happen, humans create a lot of areas under the neurocomputing umbrella:
- Artificial intelligence.
- Business intelligence.
- Computer vision.
- Deep learning.
- Internet of things
- Machine learning.
- Natural language processing.
- Neuromorphic computing.
- Pattern recognition.
- Recommender engines.
From what I’ve seen, some areas are easier to integrate if you are in a certain role already:
- Back-End.- Machine-Learning, Deep learning, recommender engines.
- Front-End.- Computer vision, Pattern recognition,
- DBA.- Business Intelligence, Data mining.
- DevOps.- Robotics, IoT.
- Testing.- RPA’s (Robotic Process Automation), A/B Testing.
This is the harder of all the software engineering pillars, and, since is harder, not too many people will involve with it. So, hear this: since there are not too many engineers doing this, and there is a market with increasing demand on this topic, the ‘Supply and Demand Law’ will do its thing and soon the engineers with skills in Neurocomputing will be the best paid.
As you can see the ROI on this topic is really high and is a trend that will become really relevant (probably even a must-have) in near future. You should definitely reach a high level on this pillar if you want to avoid becoming obsolete.
There are lots of kinds of testing:
A Software solution won’t be fully finished if we don’t test it properly. Always remember the principle: “If ain’t tested, is broken”.
Not all of those tests are needed for every software you are building, or at least not at each stage.
It will be ironic that programmers wouldn’t have thought on write code to programmatically execute these tests; that is Automation testing. There are a lot of tools online ready to help you run some tests like Google Page Speed Test. And there are some customizable test tools also known as ‘record and play’. And some others with more configurable inputs that also require no code like Jmeter.
If you need to perform more custom and/or configurable automation testing there are a lot of libraries already available for each language, i.e. Selenium.
What about systems design?
Systems design doesn’t necessarily belong to the Software Architecture area, I agree is a very important part of the Software Engineering area, for Architecture, we have Architectural patterns and some of those can also be implemented at a low level like MCV or MVVM.
Design is not a pillar per se. Is like the blueprint, the composition, the structure, or the form you give to the pillar you are building.
There are a lot of design patterns that can help you as a guide when designing a system. Is very important that you learn at least the most important ones. The most common design patterns used are: Dependency Injection, Singleton, Factory, Decorator, Facade, Repository, and Observer. But there are many more very interesting that can you help you a lot.
Do not try to reinvent the wheel, almost for every software necessity there is a design pattern already. Remember design patterns are half-cooked and you are responsible to finish cooking them at the needed level, also be aware to not overcook them (overengineer).
What did we just see? That seems like rocket science, right? Yeah, Software Engineering is a really complex career, and also consider that every day we have new frameworks, new versions, new libraries, etcetera that we need to continuously learn to always be fresh and committed to delivering the best software we can.
But don’t get it wrong, we all love our work. To tell a computer what to do and then see it in action is spectacular and it is worth all the effort. Most of us will do our work for free, but they insist on paying us. so, what we can do? ;)
If you are thinking of increasing your tech stack to become the Ultimate Software Engineer, please do it!. Do it for the world, and for all humans that just want to let computers do all the work so we can “finally rest and watch the sunrise on a grateful universe”.
If you like this article you will love this about (C)Lean Code. It will teach you to build a strong, maintainable pillar.