The truth of algorithms and why you should question the use of Big Data

Firstly, what is Big Data?

Big data is data sets that are too large or complex to be dealt with by traditional data-processing applications and typically refers to analysing and evaluating of that data.

Creating algorithms that analyse Big Data can be extremely beneficial and whether you realise it or not, it typically affects your daily life. With developments in Big Data, Transport for London (TfL) can analyse and allocate more buses to more loaded routes to minimise traffic congestion. Whilst retailers can dynamically adjust pricing on items across their inventory with respect to costs and stock.

Here are just a few examples of how businesses can benefit:

• Reduce costs – Identify cost savings opportunities in an organisational spend
• Better decision marketing – Analyse data faster than previously possible
• Improved customer service – Distinguish customer needs and their satisfaction from your product/service
• Increased agility – Understand market conditions/trends

However, as Cathy O’Neil discussed in her TED Talk, algorithms can be secret, important and harmful: “Weapons of Math Destruction”. Why? Because these algorithms that are running on big data are deciding who gets loans, who gets job interviews or who’s child gets scouted by that major league sports team.

Algorithms are man-made and taught how to learn, however by doing so they suffer inherent issues. Namely, that they are unable to absorb the data that isn’t purposefully included. Whilst they may include the bias of predecessors, that can present the results irrelevant of whether it is profiling or stereotyping.

So why should you question the use of Big Data and the algorithms?

Because whilst we’ve made huge jumps in technology, without adequate consideration into data quality, cybersecurity and compliance then it has the potential to cause huge damage to society and businesses.

—

This blog was inspired by the Cathy O’Neil – Watch the full video yourself here https://www.ted.com/talks/cathy_o_neil_the_era_of_blind_faith_in_big_data_must_end#t-703577