What is the purpose of this?

Data Engineer Blog 0x001B — Saturday Morning Coffee Thoughts: How the purpose of the work determines what your focus should be.

Og Ramos


Photo by Pablo Arroyo on Unsplash

I’ve been thinking about how I make decisions when we’re looking at requirements. For example, how does someone know what to focus on and what work should be done to solve an issue?

Client Focus

How does the issue impact the client? Looking at the problem and how it affects a client could allow you to find creative ways to solve that specific issue. For example, a long time ago, I worked with a client who had a performance issue with their reporting. It was using a long-running query that was taking ten minutes to run. They brought us in to look at the query utilized for this particular report, and we did performance tuning. We never looked at the report. Instead, we got the query and started working and tuning it. Then, after a couple of hours, the query ran in seconds, 10 seconds, precisely. Our work was done! Or was it?

A week later, we got a call that the end-users were complaining that the report was still slow. How could this be? We had indexes, well-formed SQL code, and all the bells and whistles possible. So I asked to look at the report, and the problem was there on the report. Every time someone wanted to filter the data, it would run the query; every time you tried to filter or drill down into the data, the reporting tool would rerun the query. So every click was 10 seconds, which did add up after a while of clicking.

I agree that both places needed to be verified and checked, but the solution could’ve been more straightforward if we had the complete picture. Pigeonholing us to just the query did a disservice, and the answer could’ve been different.

Looking at the end user’s perspective and pains, you can find other issues impacting a successful delivery.


How many layers do I need to go through to get to the tootsie roll center of a Tootsie Pop?



Og Ramos

Writer, Cloud Data Engineer, Business Intelligence Expert, and Data Architect —http://ogramos.com/ | https://www.linkedin.com/in/ogramos/