Lessons Learned from Software Engineering Practices

Design patterns are terrific. When I first found out about design patterns, I was blown away. Factories, builders, and singletons. Everything. I studied them all and I used them everywhere, but I was particularly fond of the strategy pattern. If someone mentioned a future feature in passing I'd make sure our code was ready at all times. Found a piece of code to make more general? Time for the strategy pattern. "This keeps us agile!", I'd almost certainly have said in triumph as I demonstrated my flexible designs to my peers. But this soon started backfired. Hard. Despite good intentions, feature roadmaps change a lot. This meant my clever generalizations and abstractions had to get torn down frequently. Or worse, it was too hard to reverse them so I had to work around them. This slowed me down substantially. My clever design pattern efforts to move faster made me move slower. Because requirements change. They do that because we get wiser about what the customer wants. It's all in order. I still use design patterns though. The strategy pattern is still a favorite. But I'm far more selective and conservative about when to use it. I prefer to start with specific solutions over general ones. Letting them evolve as we add more cases. Then, if a pattern emerges and there's enough friction to warrant a refactor: I'll go for the design pattern. The lesson: Design patterns are useful, but premature design is counter-productive. It's more than okay to wait for a strong signal. (Note: this lesson goes beyond code-level optimizations. On the system level, premature design can be 100x worse because decisions aren't as easy to reverse). #softwareengineering

People tend to believe that there is more of a trade-off between Quality and Speed than there really is. Most of the time, if you’re doing a good job of refactoring and writing tests as you go, the final product doesn’t take any longer than it would have to do it “the hacky way,” if by “the final product” we mean “a system that actually works and does what its users need.” True, once in a while there are huge problems that you can’t tackle right now. For example, maybe the right thing to do is to redesign your application with a whole new data store, but that could take years. You can’t do it right now. That said, _most_ of the problems that users experience with software don’t come from these decisions to set aside these huge projects for later. Most issues come from a series of small decisions that grew over time into a complexity that nobody understood or managed correctly. Often they come from a developer just _feeling_ rushed, like they can’t refactor this right now, they can’t test this right now, etc., when really they do have the time to do it. Let me give a specific example. I once caused a fairly significant production outage at YouTube. I was mid refactoring a very complex piece of code, and I had not written a test at the highest level to make sure all the pieces still worked well together. Had I written the test, the issue absolutely would have been caught. So egg on my face, right? Well, there are two lessons to take away from this, actually: 1. The shallowest lesson here is that one should have tests that cover what one is refactoring. 2. A deeper lesson is this: the function I was refactoring had grown incrementally into a hairball that was basically impossible to understand. Every person who had worked on it (many people) had chosen _not_ to fix its structure over a period of many years. They also had never written a test for its behavior—you couldn’t even look at a test to understand what it was _supposed_ to do. The function was basically Pandora’s Box; some day somebody was going to touch it and cause something to break, and the person who did that happened to be me, because I wrote about 100 changes to it (incremental refactorings) and one of them happened to have a bug. So should I have written a test? Maybe. Honestly, until I went through the process of refactoring it, it was really hard to tell what it was _supposed_ to do in all sorts of various edge cases that it covered. Am I personally responsible for that outage? Of course! But am I the _only_ person who contributed to that outage occurring? No. Every time somebody says, “well, I’ll refactor that later,” and doesn’t, “we really ought to have monitoring eventually,” “I should write some tests some day,” THAT is where most problems start. And the wild thing is—almost always, those problems didn’t have to start! If everybody had maintained the system well as they went, it’s unlikely there would have been any impact to the speed of delivery.

You learn life's most impactful lessons from your mistakes and failures. Isn't it? However, when it happens, what goes in the mind? You are usually beating yourself up, "How did I do it?" "Why I couldn't be more mindful?" But did you know how we respond to them, reshapes our path? A few years ago, I was spearheading a project – a new project management software developed for a pro-bono consulting gig. We meticulously crafted each feature, aiming to create a budget-friendly yet powerful tool. The team was charged, and expectations were soaring. Then came the launch, and with it, a reality check. Our vision of a feature-packed tool had unintentionally led us astray from what the end-users actually needed - simplicity and intuitiveness. As the adoption rate dwindled and the discontent amongst the users grew, it dawned on us that we had been so engrossed in the technicality of the product that we overlooked the paramount importance of user experience. This realization was a significant turning point in my professional journey. Two valuable lessons were learned: 1️⃣ First, every aspect of product development must be guided by the user’s needs and experiences. 2️⃣ Second, even the most technologically advanced product will fail if it is not user-friendly. With these insights, we recalibrated our development process. We introduced a user-centric approach, involving potential users early in the design phase, and emphasized the role of user testing and feedback. The results? Products that not only met but frequently exceeded customer expectations, elevating our market share and customer satisfaction scores. This experience has made me a fervent advocate for customer involvement throughout the development process. It reiterated to me that we aren't just creating products; we are shaping experiences. Now, I encourage you to reflect on your professional journey. Is there a setback that taught you invaluable lessons? And more importantly, how have you applied those learnings to foster growth and improvement? Looking forward to hearing your stories in comments!

Has anybody had projects go wrong? I’ll be the first to raise my hand! In my time working with AI, I’ve had my fair share of projects that didn’t go as planned. And while it wasn’t the experience I thought I wanted, the unexpected outcomes always led to lessons and opportunities for growth. So here are some of the things I’ve learned: I learned to embrace the complexity. AI systems operate in complex, often unpredictable environments. Our experience taught us to embrace this complexity, not shy away from it. It pushed us to develop more robust testing scenarios that mimic real-world complexities more accurately. Second, data quality is paramount. Data quality is of critical importance. It's not just about quantity; the relevance, accuracy, and diversity of data play a pivotal role in the success of AI models. Next, algorithmic transparency. Understanding why an AI model makes certain decisions is crucial. Our project's challenges highlighted the need for greater transparency in our algorithms, propelling us towards adopting explainable AI practices. Another very important lesson is the power of team resilience. Facing setbacks tested our team's resilience but also brought us closer. It fostered a culture where every member feels valued and empowered to share ideas and concerns openly, driving collective problem-solving and innovation. Lastly, continuous learning is key. When projects go wrong, it reinforces the idea that in the AI field, continuous learning and adaptation are not just beneficial but necessary. It's a journey of constant evolution, where each setback can lead to greater achievements. In the world of AI, ‘failure’ is not a setback, it’s supposed to be used for growth and innovation! What are some lessons that you’ve learned from unexpected outcomes? #innovation #failures #lessonslearned #AI #technology #Kibsi #computervision #growth #development

2 weeks ago I made a minor code change that broke one of our entire features. Here’s what happened: Every so often the maintenance team will scan through repos asking feature teams to remove unused code, resolve security bugs, update packages, etc. I received an email from them saying our product has a few feature flights listed in a repo and to remove them. Then further down it listed instructions on how to make sure all of the flight references are deleted if unused. I already had my main work to do, but they were asking me to fix this bug in the next couple of days. Seemed like a simple change right? So I quickly skimmed the instructions in the bug and removed the flights in the listed repository, but didn’t read the instructions further down “to remove the flights from all other repos”. I tested the changes immediately in our beta test environments and encountered no issues, so I went back to my main work. 3 weeks later, I received an email from the test vendor that filed 5 breaking bugs in our pre-prod environment… and this is like 5 days before we ship to prod. The reason this wasn’t caught in earlier test environments is that the UX has this flight enabled for all users. So no accounts encountered issues. In the more mature test environment( canary) and production, there is a flight check reference and the flight wasn’t turned on for all users. So it was checking for a flight that I already deleted from the service side which resulted in those test users hitting a breaking experience. Luckily I quickly diagnosed the issue and reverted the change I made. A couple of lessons here: 1) Even the smallest changes can have huge impacts on the product. Don’t underestimate them. 2) It’s better to find dedicated time (even if it’s a day later) to do a task with your FULL attention than to rush and possibly miss something. #softwareengineering #softwaretesting #milestones

I was a good software engineer, but I wasn't great. Then, I had an epiphany. It came while observing my Tech Lead (at the time), and wondering what made them so special. They were strong technically, but not exceptionally so. Yet they always seemed to have all the answers. So I started observing them more closely. And I eventually realized what made them special: 👉 They understood how the business worked. Just as well as they understood the code. That's it. So simple. But also... difficult. It's difficult for us to switch from "tech mode" to "business mode". It's a different school of thought. We're taught to code. We're taught to design scalable systems. We're taught to read stack traces and debug. We're not taught to ask questions like, "Why does sales tax need to be estimated in the cart preview?" (This was my Tech Leads's question that led to my epiphany 😄) But learning to ask these questions is what it takes to be the best. 👉 The best engineers develop a deep understanding of the products they build. Their product knowledge guides their technical designs, making them natural leaders, planners, and candidates for promotion. Invest in your product knowledge. You'll thank yourself later. #softwareengineering #product #professionaldevelopment #leadership