Optimize Late and Not Often

Many software engineers, myself included, are driven to build the best software possible, pouring their creative juices into developing “the perfect” solution. This intention is admirable and it is this quality that can lead to extremely innovative applications. However, I believe it is also this perfectionist quality that can sometimes cause us to lose sight of the bigger picture, our highest priorities. System optimization is one of those “quagmire” areas where we can easily lose sight of reality, as well as our priorities. We can fall into the habit of allowing ourselves to believe that every line of code we write must perform at blazing speed or use the absolute minimum system resources. I’ve found myself, on occasion, after researching optimization patterns or coding and recoding a particular algorithm for hours or days, when suddenly I think, “What is the problem with my original implementation? Why do I think this code won’t perform well enough? What performance tests did I run, against what performance requirement, that pointed me to this code being an issue?”. My job, my responsibility, my goal, is to deliver value and value can only be derived from a real need. When I allow myself to go off on “optimization tangents” (or any sort of effort that could be challenged as being “over-engineering”, for that matter), I lose valuable development cycles – cycles that could have been used to deliver something that actually provides a tangible benefit. The tendency towards over-engineering is natural – I’ll guarantee that we all do it. And so we must accept that, become aware of it and not beat ourselves up when we catch ourselves doing it. One way we can maintain this awareness is to form a habit of frequently asking ourselves this question: “What is the real value in doing the task I’m doing at this moment? Is it real value, or is it imagined?”

Let’s focus specifically on software optimization. What are some of the consequences of taking the “optimize it, optimize it all!” approach?

  • In the time I spent optimizing code (that may already be quite fast enough), I might have implemented and deployed a new feature that could really help the end user.
  • Optimization very often results in code that is more difficult to understand and maintain.

Read more ›

Tagged with: , ,
Posted in Java Programming, Software Development, Testing

Dependency Injection and Loose Coupling: How They Affect Your Ability to Test

Even though the concepts of “dependency injection” and “loose coupling” have been advocated and written about for at least the past decade, I’m still routinely finding examples that make me think, on the whole, we developers may be underestimating the power of these simple ideas and their related practices. In this post, I’ll describe a real-world, simple scenario I recently encountered and point out how a few very slight changes in thinking could have significantly improved the design and avoided a lot of challenges down the road. I’ll provide some sample code that illustrates why ignoring these principles can get us into trouble, as well as sample code that illustrates how we can clean up our design and make our lives so much easier.

I recently ran into this issue on a project… I found a need to update a Java class that represented a scheduled “job” that retrieved a very large file (contained hundreds of millions lines/records) from a remote URL and processed that file. In a nutshell, here are the primary responsibilities for this class:

  1. Download file from a URL to the local filesystem.
  2. Parse each line contained in the downloaded file, checking to see whether the line represents an entry the application is interested in.
  3. For each line of interest, update the application’s database – depending on the line’s content, that line would either be ignored, or would flag the class to delete existing records or add new records to several tables in the application’s database.

Read more ›

Posted in Java Programming, Object-Oriented Design, Software Development

Software Project Process: Focus on Core Beliefs

For the experienced software developers out there, I have a question: what are the main things you believe, based on your experience, most influence the success of a software project? I should provide some context. I’m not talking about a trivial project, or a throw-away prototype, although I’m sure we all recognize that there are some principles and practices that apply to all software projects – for all projects of any type, for that matter. However, for this discussion, consider projects whose goal is to produce a production-quality enterprise-scale application. In a previous post – Wise Project Management Always Starts With “Why?” – I said that I believe we can only be effective in execution of any project when we first understand why we need to do certain things and everyone understands the “why”. This can help us to keep our focus on value – if we can’t articulate the “why” behind what we’re planning to spend significant time and effort on, there’s a red flag that maybe we shouldn’t be doing it, or should at most be spending minimal effort on it.

In this post, I’d like to consider this idea from a slightly more concrete perspective. Consider some of the core beliefs you hold about what can make or break a software project. I hope issues such as “which line the team decides the curly braces should be placed on” are not in this list for you. I’m talking about issues that really impact the team morale, the ability to deliver on agreed quality and schedule, to keep the project moving down the tracks in a positive direction, with clear intention. Read more ›

Tagged with: ,
Posted in Agile Software Development, Project Management

Map Your Mind: Mind Maps as an Alternative for Capturing Ideas

Do you ever have so many ideas running through your head, flip-flopping, churning, bumping into each other, that you feel that your head will explode? Do you often get stuck while trying to think through the aspects of a new idea, the steps of a process, planning a new project? Or, maybe you’ve been taking notes during meetings and find that, unless you write down every word that’s said, you just cannot recall the main ideas and action items from the meeting.

We’ve been trained to attempt to capture the content of our rapidly moving brains in mainly linear ways. Think of how most of us take notes in a meeting. We take a lined note pad, try to write down ideas as they’re stated, and we may try to somehow categorize them after the fact. It would be nice if our brains would follow a single line of thinking to its conclusion before being distracted by a totally unrelated thought. I’d personally love meetings in which all participants could diligently follow the established agenda, never veering off into wild tangents, or jumping back and forth between topics. I say that, but I can imagine how boring that would likely be. The fact is that humans are not computers and our brains will never be that disciplined. Our unruly brains seem to have no shame. They fire thoughts in a seemingly random fashion – one second thinking about the plan for our upcoming project, the next about our upcoming Hawaiian vacation. Read more ›

Tagged with: , ,
Posted in Mind Mapping, Project Documentation

Quick Intro to Hibernate Search and Lucene

In my previous post “Adding the Power of Search to Your Hibernate App – The Easy Way“, I talked a little about when you may want to consider integrating a search capability into your application using Hibernate Search, as well as a bit about Hibernate Search and how it relates to Hibernate Core, Lucene and Solr. In this post, we’re going to take a quick look at a sample application (really, it’s a JUnit test case) that uses Hibernate Core, with Java Persistence API (JPA) annotations, to persist a simple entity to a relational database and Hibernate Search to run some searches against the Lucene indexes created/updated as the Hibernate-managed entities are updated in the database.  Read more ›

Tagged with: , , , , ,
Posted in Database Programming, Java Programming, Software Development

Adding the Power of Search to Your Hibernate App – The Easy Way

I’m currently working on a software project whose data layer is built using Hibernate – an Object-Relational Mapping (ORM) framework that takes a lot of the tedious work out of tying a Java-based application to a relational database. We recently found that the system needed to expose a full-text search mechanism for a subset of the data being stored in its backend Oracle database. Our first instinct was to deploy a standalone instance of the Solr search platform, which is built on top of the very popular and mature Lucene search engine library. We’d then have our core application call out to the Solr engine remotely and have Solr return search results in JSON format. If you’re an experienced Java developer, I have no doubt you’ve at least heard of all these technologies.

I’ve never personally had a need to implement full-text search. However, I was familiar with basic full-text search concepts and I did know that Solr was one of the more popular search options. I was casually chatting with one of my colleagues – Eddie – about his project and he mentioned that his team had selected Hibernate Search as their search solution. Read more ›

Tagged with: , , ,
Posted in Database Programming, Java Programming, Software Development

Wise Project Management Always Starts With “Why?”

Why-how-what
As I wander from project to project in my career as a software developer, I notice that, despite the remarkable progress we’ve made in technologies and technical approaches over the past decade, one area still seems to give us fits. We love our technology, we love writing code… but, the idea of living by a “process” gives us the “bad sushi” feeling. We’ve lived under the tyranny of “Process Overlords”, whose wisdom we clearly did not understand – especially since the overlords’ grand plans seldom seems to have resulted in something we could bring ourselves to call “a success”. So, we’ve been hurt before. I think we could use a hug now. Read more ›

Tagged with: , , ,
Posted in Agile Software Development, Project Management
Follow

Get every new post delivered to your Inbox.