ETL- Is it an art or a science? And which should it be?

Science when we start building a rigid framework for it in which it has to be done this one and only one way. The funny thing with the latter bit is exceptions to the latter rule invariably occur since a company's data is of an irregular nature and soon enough the purported advantages like consistency of approach no longer compel.

Art when executed with a certain flair; otherwise it's simply the developer's whim disguised as art. As for the flair itself, let's just call it a set of of one or more guiding principles or if you dig industry parlance, the phrase "best practice" may come to mind although best practices are collective in nature and a guiding principle tends to be individual specific. In yours truly case, I go by what I termed the principle of non-repetitive reads (NR²), i.e. frequently consumed data is to be read only once, the strict abidance of which automatically results in a “lean” ETL process with a compact form factor.

Apart from building a "mean" ETL engine, why else should organizations pursue ETL as an art form? For the designer, ETL is suddenly "fun" as creativity permeates the picture, resulting in additional avenues for downstream implementation, i.e. the developer can now interpret the design in further ways than before, much like a connoisseur fathoming an art piece. Intangible benefits to be sure but nonetheless powerful employee motivators and retention factors.

The meaning of life is to attain happiness and the route to it is by producing competitive values for others. A value is most desired when it is unique - just look at prices for one of a kind pieces of art. Put the two together and you have staff happily producing non-bloated custom solutions away without their managers' coercion and with no contemplations of greener pastures. What can better that?