Innovation and Efficiency

A week or so ago a McKinsey article on innovation popped up in my LinkedIn feed. It’s an old article but probably still relevant today. It’s the usual lament about how difficult it is to innovate. My opinion is that innovation flies in the face of today’s efficiency paradigm. Being innovative or creative is very inefficient. It takes a lot of trial and errors and really creative ideas just don’t pop up on demand. You can’t schedule when you are going to solve a problem – you solve it when you solve it.

My experience with coming up with visuals to go along with my posts demonstrates the difficulties. If I do come up with a visual idea, and that is not always the case, it takes a while to draw or paint the visual idea I have in mind. Sometimes I do photo manipulation of some drawing I already have and sometimes it’s a completely new drawing. Sometimes it’s abstract and sometimes it’s crap. There is nothing efficient about the process.

Same thing with developing macros or designing in Power BI. It takes forever to create a macro or to design the Power BI. I can create re-usable macro modules that I copy from program to program but it is still a slog to make it work, especially if I have a vision in my head of how it is to work. But once I’ve created the macro and have done several rounds of using the macro, it becomes just a matter of pushing a button. The macro becomes the efficient thing but creating the macro is inefficient.

Power BI is turning out to be the same thing. Creating the visual for your Power BI is time consuming but I imagine that once you’ve created the visuals and use it a while, the updating of your visuals becomes automatic and efficient. Automating the updates of dashboards, KPIs, or reports becomes efficient, creating them is inefficient.

But we live in a world that is all about efficiency and speed. Instant gratification. There is a conflict between wanting to be innovative and wanting efficiency and management haven’t quite caught on to that fact.

Right now, I’m reading a book The Efficiency Paradox – What Big Data Can’t Do by Edward Tenner and he pretty much says the same thing: creativity and innovation is messy and inefficient. And sometimes being inefficient is better.

For instance, taking notes by hand rather than typing in the lecture verbatim is much better if you want to understand the concepts being taught in class. Writing, which is inefficient, forces you to think about the concepts and encapsulate the them in your own words. It forces you to think.

By the way, I was wondering why I wasn’t getting much posting done during this holiday weekend, then I realized I did some work on Friday morning and Saturday afternoon and onto the night – very late at night. Friday was to develop a macro to get some data and Saturday was to work on the Power BI data analytics idea I had and produce the relevant visuals. Both of these activities require some problem solving and creativity/designing, and they just take a long time. They sucked up the time that could have been devoted to drawing and writing my posts.

All of this is to say these things just take time. An automated program might have done it quickly but it might have produced the same ol’ same ol’.

So now I feel better that I haven’t posted as much as I had hoped this weekend.