International Institute for Analytics

The vast majority of people building analytics and data science processes have every intention of being good and ethical. As a result, most potentially unethical and evil processes arise in situations where that wasn’t the intention. The problem is typically that proper focus and governance is not in place to keep analytics and data science processes on the side of good. On top of that, what is good and what is evil isn’t nearly as clear cut as we’d wish it to be.

We’ve had technical people focused on the ingestion and management of data for decades. But, only recently has data engineering become a critical, widespread role. Why is that? This post will outline a somewhat contrarian view as to why data engineering has become a critical function and how we might expect the role to evolve over time.

Lately I’ve had a lot of conversations with clients about the intersection of ethics and analytics. I’ve also been presenting on the topic at a number of conferences. The interest in ethics has exploded recently, driven in large part by the rise of artificial intelligence. One common question I get is what my top tip would be for a company to get started in becoming a leader in analytical ethics. I’ll discuss my answer in this post: intentionality.

Not long ago, the role of Data Scientist was what most companies wanted to discuss with me in terms of roles they needed to understand and add to their organizations. Then, the role of Data Engineer became a big topic of discussion. In the past year, there has been a massive increase of attention being paid to yet another role that is still new enough that its title hasn’t been standardized. This role is referred to by a range of names from Analytics Translator, to Analytics Catalyst, to Analytics Liaison, and more.

Historically, concerns about over-zealous censorship have focused on repressive governments. In recent times, however, a new path to censorship has arisen in the form of search engine and social media companies that are building analytically-based censorship algorithms. This post outlines why using analytics for centralized censorship is a steep and slippery slope and also lay out an alternative that will enable those same censorship analytics to provide people with a choice rather than a dictate.

There is a lot of well-deserved hype for artificial intelligence algorithms and for deep learning in specific. Self-driving vehicles are already being tested and rolled out into our communities. So, the future is here. The way the cars are enabled is partly through using convolutional neural networks to do object detection. There are certainly many other algorithms that are part of the self-driving process, but a lot of the key algorithms that enabled us to get to where we are today are the convolutional neural networks that are explained in this research brief.

While there are many advantages to the cloud, it is also necessary to use caution to make sure that the risks of the cloud are mitigated while pursuing the advantages.

As data science and analytics teams continue to feel pressure to deliver more value from analytics, many organizations still struggle with the processes and technology required to deploy models into production and more rapidly make data-driven decisions. When evaluating how to best undertake these activities, organizations should consider an important distinction to determine the best path forward.

Over the years, I’ve seen analytics professionals of all stripes blow their credibility and lessen their impact by falling into a common trap. I have to admit that I fell victim to the same trap early in my career. While our intentions are pure, our analytical minds and approaches can get the best of us and we explain too much. We’ll be better off if we learn to provide less detail and stop talking sooner than we are naturally inclined to.

The analytics landscape has matured, broadened, and totally transformed in recent years. This transformation has been driven by a wide range of factors including big data, the cloud, the increase in processing capability, the decrease in storage and processing costs, and the near-ubiquitous availability of algorithms. As a result, an organization must also totally transform the tools and processes it uses for analytics if it is to recognize its analytics potential.