Monthly Archives: July 2020

Experts from MIT and IBM held a webinar this week to discuss where AI technologies are today and advances that will help make their usage more practical and widespread.

Image: Sompong Rattanakunchon / Getty Images

Artificial intelligence has made significant strides in recent years, but modern AI techniques remain limited, a panel of MIT professors and IBM's director of the Watson AI Lab said during a webinar this week.

Neural networks can perform specific, well-defined tasks but they struggle in real-world situations that go beyond pattern recognition and present obstacles like limited data, reliance on self-training, and answering questions like "why" and "how" versus "what," the panel said.

The future of AI depends on enabling AI systems to do something once considered impossible: Learn by demonstrating flexibility, some semblance of reasoning, and/or by transferring knowledge from one set of tasks to another, the group said.

SEE: Robotic process automation: A cheat sheet (free PDF) (TechRepublic)

The panel discussion was moderated by David Schubmehl, a research director at IDC, and it began with a question he posed asking about the current limitations of AI and machine learning.

"The striking success right now in particular, in machine learning, is in problems that require interpretation of signalsimages, speech and language," said panelist Leslie Kaelbling, a computer science and engineering professor at MIT.

For years, people have tried to solve problems like detecting faces and images and directly engineering solutions that didn't work, she said.

We have become good at engineering algorithms that take data and use that to derive a solution, she said. "That's been an amazing success." But it takes a lot of data and a lot of computation so for some problems formulations aren't available yet that would let us learn from the amount of data available, Kaelbling said.

SEE:9 super-smart problem solvers take on bias in AI, microplastics, and language lessons for chatbots(TechRepublic)

One of her areas of focus is in robotics, and it's harder to get training examples there because robots are expensive and parts break, "so we really have to be able to learn from smaller amounts of data," Kaelbling said.

Neural networks and deep learning are the "latest and greatest way to frame those sorts of problems and the successes are many," added Josh Tenenbaum, a professor of cognitive science and computation at MIT.

But when talking about general intelligence and how to get machines to understand the world there is still a huge gap, he said.

"But on the research side really exciting things are starting to happen to try to capture some steps to more general forms of intelligence [in] machines," he said. In his work, "we're seeing ways in which we can draw insights from how humans understand the world and taking small steps to put them in machines."

Although people think of AI as being synonymous with automation, it is incredibly labor intensive in a way that doesn't work for most of the problems we want to solve, noted David Cox, IBM director of the MIT-IBM Watson AI Lab.

Echoing Kaelbling, Cox said that leveraging tools today like deep learning requires huge amounts of "carefully curated, bias-balanced data," to be able to use them well. Additionally, for most problems we are trying to solve, we don't have those "giant rivers of data" to build a dam in front of to extract some value from that river, Cox said.

Today, companies are more focused on solving some type of one-off problem and even when they have big data, it's rarely curated, he said. "So most of the problems we love to solve with AIwe don't have the right tools for that."

That's because we have problems with bias and interpretability with humans using these tools and they have to understand why they are making these decisions, Cox said. "They're all barriers."

However, he said, there's enormous opportunity looking at all these different fields to chart a path forward.

That includes using deep learning, which is good for pattern recognition, to help solve difficult search problems, Tenenbaum said.To develop intelligent agents, scientists need to use all the available tools, said Kaelbling. For example, neural networks are needed for perception as well as higher level and more abstract types of reasoning to decide, for example, what to make for dinner or to decide how to disperse supplies.

"The critical thing technologically is to realize the sweet spot for each piece and figure out what it is good at and not good at. Scientists need to understand the role each piece plays," she said.

The MIT and IBM AI experts also discussed a new foundational method known as neurosymbolic AI, which is the ability to combine statistical, data-driven learning of neural networks with the powerful knowledge representation and reasoning of symbolic approaches.

Moderator Schubmehl commented that having a combination of neurosymbolic AI and deep learning "might really be the holy grail" for advancing real-world AI.

Kaelbling agreed, adding that it may be not just those two techniques but include others as well.

One of the themes that emerged from the webinar is that there is a very helpful confluence of all types of AI that are now being used, said Cox. The next evolution of very practical AI is going to be understanding the science of finding things and building a system we can reason with and grow and learn from, and determine what is going to happen. "That will be when AI hits its stride," he said.

Learn the latest news and best practices about data science, big data analytics, and artificial intelligence. Delivered Mondays

Link:

The path to real-world artificial intelligence - TechRepublic

Take the latest VB Survey to share how your company is implementing AI today.

Researchers from Googles DeepMind and the University of Oxford recommend that AI practitioners draw on decolonial theory to reform the industry, put ethical principles into practice, and avoid further algorithmic exploitation or oppression.

The researchers detailed how to build AI systems while critically examining colonialism and colonial forms of AI already in use in a preprint paper released Thursday. The paper was coauthored by DeepMind research scientists William Isaac and Shakir Mohammed and Marie-Therese Png, an Oxford doctoral student and DeepMind Ethics and Society intern who previously provided tech advice to the United Nations.

The researchers posit that power is at the heart of ethics debates and that conversations about power are incomplete if they do not include historical context and recognize the structural legacy of colonialism that continues to inform power dynamics today. They further argue that inequities like racial capitalism, class inequality, and heteronormative patriarchy have roots in colonialism and that we need to recognize these power dynamics when designing AI systems to avoid perpetuating such harms.

Any commitment to building the responsible and beneficial AI of the future ties us to the hierarchies, philosophy, and technology inherited from the past, and a renewed responsibility to the technology of the present, the paper reads. This is needed in order to better align our research and technology development with established and emerging ethical principles and regulation, and to empower vulnerable peoples who, so often, bear the brunt of negative impacts of innovation and scientific progress.

The paper incorporates a range of suggestions, such as analyzing data colonialism and decolonization of data relationshipsand employing the critical technical approach to AI development Philip Agre proposed in 1997.

The notion of anticolonial AI builds on a growing body of AI research that stresses the importance of including feedback from people most impacted by AI systems. An article released in Nature earlier this week argues that the AI community must ask how systems shift power and asserts that an indifferent field serves the powerful. VentureBeat explored how power shapes AI ethics in a special issue last fall. Power dynamics were also a main topic of discussion at the ACM FAccT conference held in early 2020 as more businesses and national governments consider how to put AI ethics principles into practice.

The DeepMind paper interrogates how colonial features are found in algorithmic decision-making systems and what the authors call sites of coloniality, or practices that can perpetuate colonial AI. These include beta testing on disadvantaged communities like Cambridge Analytica conducting tests in Kenya and Nigeria or Palantir using predictive policing to target Black residents of New Orleans. Theres also ghost work, the practice of relying on low-wage workers for data labeling and AI system development. Some argue ghost work can lead to the creation of a new global underclass.

The authors define algorithmic exploitation as the ways institutions or businesses use algorithms to take advantage of already marginalized people and algorithmic oppression as the subordination of a group of people and privileging of another through the use of automation or data-driven predictive systems.

Ethics principles from groups like G20 and OECD feature in the paper, as well as issues like AI nationalism and the rise of the U.S. and China as AI superpowers.

Power imbalances within the global AI governance discourse encompasses issues of data inequality and data infrastructure sovereignty, but also extends beyond this. We must contend with questions of who any AI regulatory norms and standards are protecting, who is empowered to project these norms, and the risks posed by a minority continuing to benefit from the centralization of power and capital through mechanisms of dispossession, the paper reads. Tactics the authors recommend include political community action, critical technical practice, and drawing on past examples of resistance and recovery from colonialist systems.

A number of members of the AI ethics community, from relational ethics researcher Abeba Birhane to Partnership on AI, have called on machine learning practitioners to place people who are most impacted by algorithmic systems at the center of development processes. The paper explores concepts similar to those in a recent paper about how to combat anti-Blackness in the AI community, Ruha Benjamins concept of abolitionist tools, and ideas of emancipatory AI.

The authors also incorporate a sentiment expressed in an open letter Black members of the AI and computing community released last month during Black Lives Matter protests, which asks AI practitioners to recognize the ways their creations may support racism and systemic oppression in areas like housing, education, health care, and employment.

See the original post:

DeepMind researchers propose rebuilding the AI industry on a base of anticolonialism - VentureBeat

Every department in a company has its own challenges.

In the case of Human Resources, recruitment and onboarding processes, employee orientations, process paperwork, and background checks is a handful and many a time painstaking mostly because of the repetitive and manual nature of the work. The most challenging of all is engaging with employees on human grounds to understand their needs.

As leaders today are observing the AI revolution across every process, Human resources is no exception: there has been a visible wave of AI disruption across HR functions. According to an IBMs survey from 2017, among 6000 executives, 66% of CEOs believe that cognitive computing can drive compelling value in HR while half of the HR personnel believe this may affect roles in the HR organization. The study clearly exhibits the apprehension of HR executives caused by the AI disruption in their field.

While one aspect of AI is creating uneasiness: the other is promising convenience. AI aims to empower the HR department with the right knowledge to optimize processes with less manual power and guarantees to mitigate errors.

TheCOVID-19 pandemic has highlighted thepower of AIin real-time< Backlink-https://us.sganalytics.com/blog/ai-can-detect-infections-with-96-percent-accuracy-can-ai-predict-the-next-pandemic/>, including its shortcomings. At the crux of the AI evolution is the minimization of human labored processes. Sophisticated AI algorithms can analyze large amounts of data in no time and self-educate themselves to recognize and map patterns, which can come in handy for HR staffs to plan and operate strategically.

While a human can be biased, get bored and make unintended mistakes provoking inadequacy in productivity and efficiency, AI programs are unbiased and diligent, enabling more productivity and efficiency.

HR executives who perform tasks like applicant tracking, payroll, training, and job postings manually without automation, state that they spend 14 hours a week on an average on these tasks. Leveraging AI to automate these HR processes can be extremely pertinent for meeting the following key business requirements: First, save time and increase efficiency; Second, provide real-time responses and solutions that meet employee expectations.

As per a Mckinseys study AI will drastically change business regardless of the industry. AI could potentially deliver an additional economic output of around $13 trillion by 2030, boosting global GDP by about 1.2 percent a year.

Lets dive deeper to understand how AI can help sophisticate HR processes while not necessarily replacing human resource personnel.

1. Improved Employee Experience

Employees are the first customers for any organization. Hence employee experience is as important as customer experience.

As employee experience is becoming the next competitive edge for businesses, the coming days will be focused on providing personalized engagement and improving employee experience for human resources.

According to a Deloitte survey, 80% HR executives rate employee experience as important, while only 22% believe their organization excels at providing a differentiated employee experience.

Additionally, the advent of smart workplace has raised the bars of employees expectations for work-space experience and engagement factors.

Jennifer Stroud, HR Evangelist & Transformation Leader atServiceNow, says,We have seen the need for chatbots, AI and machine learning in the workplace to drive more productivity as well as modern, consumerized employee experiences. These consumer technology solutions are exactly what employees want in the workplace.

Engaging AI can help the HR department provide personalized employee engagement experiences across the entire employee lifecycle, right from recruitment and onboarding to career pathing.

2. Empowering HRs to make Data-Driven decisions

In common, the data-to-decision workflow looks like the below figure for many people.

Source: jobsoffice

Many HR technologies still follow the above workflow and depend on manual methods to glean insights from data. This task grows tedious and creates a bottleneck for end-users (data analysts) to draw insights within the stipulated time leading to decision making on outdated data.

While frontier technologies like data analytics are advancing to provide real-time data to make fast and fact-based decisions, AI can assist Human Resource professionals in harnessing this real-time data and making quick, consistent, and data-driven decisions. After all, the bottom line of HR agility is decision making.

3. Intelligent Automation

Intelligent automation fuses automation with AI. This will enable machines to make human-like decisions by self-educating themselves. Apart from augmenting productivity and efficiency for repetitive manual processes, this can help remove human interventions deployed for automated process completely.

1.More work in less time!

Crafting job descriptions for a particular role, filtering resumes and analyzing skillsets to find the apt talent is not only tiring and tedious, but also tricky for human resource professionals as a simple overlooked aspect can lead to a significant mistake, which may cost the company dearly in the long run. Well, AI can help HR staff overcome such scenarios by crafting bespoken job descriptions automatically and assist them in reading through thousands of resumes within a short time, thus effectively reducing the time and manual hard work put in by recruiters.

2. Identify the right talent without bias

HR personnel are humans and are likely to exhibit bias subconsciously. AI, on the other hand, is immune to human emotions which makes it the perfect fit to process candidate profiles based on required skillset without any disregard for candidates age, race, gender, geographic areas or organizational relationship. An unbiased recruitment is a win-win for both HR staff and organization. Furthermore, AI can be instrumental in increasing retention rates and establishing cultural diversity.

Consider programs like Texito, they help recognize gender bias in ads enabling recruiters to embrace a neutral language.

3. Streamline employee onboarding

The first day of an employee in an organization is like the first day of a transferred student in a new school. Although employees are grown-ups and possess the cognitive intelligence to adapt easily to an environment, deep down they look for a guidance to help them settle down in a new environment. Fortunately, organizations have HR staff to do this job. Employees generally have numerous queries on their first day regarding company policies, leaves, compensations, notice period, insurance claims, etc. As intriguing as the questions may be for an employee, these queries may turn repetitive and exhausting for an HR personnel over the time. Engaging AI chatbots makes it simple to answer such repetitive questions and make more time for the HR staff to concentrate on other essential tasks.

4. Optimize employee engagement to build better relationships

Apart from recruitment and onboarding, AI can be used to streamline processes like scheduling meetings, training employees and other such business processes. AIs capabilities to recognize personas will help Human resources professionals understand the human aspect of every single employee in-depth and enable them to shape a friendly and exciting company culture to provide unique and personalized employee engagement experiences.

5. Manage employee churn

Understanding factors that cause and arrest employee churn is the toughest part of an HRs job. People change jobs for various reasons like financial growth, career growth, shift in profiles, unsatisfied work environment, etc. Leveraging AI capabilities can help the HR department in continuously monitoring and evaluating employees thoughts about the organization, work culture, the degree of satisfaction with their job, etc. Knowing what offends or drives an employee can help in underlining the employee churn factors precisely. AI can help HR executives in performing this task more precisely.

All said and done, even though AIs capabilities would help reduce manual work and boost efficiency and productivity, artificial intelligence doesnt possess the emotional intelligence of humans. AI also cannot compensate for the humane connection that HR personnel form with employees and leverage to drive engagement and responsiveness.

Therefore, to answer the critical question that haunts HR executives Will AI be the reason why I might lose my job? No. Not really. The whole idea of AI in HR is the integration of technology to automate the more monotonous HR related tasks and optimize processes to add value to human work in less time. In the AI era, new jobs will evolve that will have new skills requirements unleashing the evolution of the HR function in an AI-first world.

Author Detail:

Jency is a technology content writer with SG Analytics. She contributes to the companys advancements by writing creative and engaging for their website and blogs. Her hobbies include music, reading, and trekking.

Company designation: Content Writer, SG Analytics

Location: Pune

Links to my blogs: https://us.sganalytics.com/blog/75-percent-consumers-anticipate-financial-impact-effects-of-covid-on-consumer-behaviour/, https://us.sganalytics.com/blog/social-media-analytics-is-truly-a-game-changer-heres-why/

Social media profile: LinkedIn https://www.linkedin.com/in/jency-durairaj-21225aa9

Excerpt from:

AI In Human Resources To AI or not to AI? - Analytics Insight

Artificial intelligence (AI) presents an opportunity to transform how we allocate credit and risk, and to create fairer, more inclusive systems. AIs ability to avoid the traditional credit reporting and scoring system that helps perpetuate existing bias makes it a rare, if not unique, opportunity to alter the status quo. However, AI can easily go in the other direction to exacerbate existing bias, creating cycles that reinforce biased credit allocation while making discrimination in lending even harder to find. Will we unlock the positive, worsen the negative, or maintain the status quo by embracing new technology?

This paper proposes a framework to evaluate the impact of AI in consumer lending. The goal is to incorporate new data and harness AI to expand credit to consumers who need it on better terms than are currently provided. It builds on our existing systems dual goals of pricing financial services based on the true risk the individual consumer poses while aiming to prevent discrimination (e.g., race, gender, DNA, marital status, etc.). This paper also provides a set of potential trade-offs for policymakers, industry and consumer advocates, technologists, and regulators to debate the tensions inherent in protecting against discrimination in a risk-based pricing system layered on top of a society with centuries of institutional discrimination.

AI is frequently discussed and ill defined. Within the world of finance, AI represents three distinct concepts: big data, machine learning, and artificial intelligence itself. Each of these has recently become feasible with advances in data generation, collection, usage, computing power, and programing. Advances in data generation are staggering: 90% of the worlds data today were generated in the past two years, IBM boldly stated. To set parameters of this discussion, below I briefly define each key term with respect to lending.

Big data fosters the inclusion of new and large-scale information not generally present in existing financial models. In consumer credit, for example, new information beyond the typical credit-reporting/credit-scoring model is often referred to by the most common credit-scoring system, FICO. This can include data points, such as payment of rent and utility bills, and personal habits, such as whether you shop at Target or Whole Foods and own a Mac or a PC, and social media data.

Machine learning (ML) occurs when computers optimize data (standard and/or big data) based on relationships they find without the traditional, more prescriptive algorithm. ML can determine new relationships that a person would never think to test: Does the type of yogurt you eat correlate with your likelihood of paying back a loan? Whether these relationships have casual properties or are only proxies for other correlated factors are critical questions in determining the legality and ethics of using ML. However, they are not relevant to the machine in solving the equation.

What constitutes true AI is still being debated, but for purposes of understanding its impact on the allocation of credit and risk, lets use the term AI to mean the inclusion of big data, machine learning, and the next step when ML becomes AI. One bank executive helpfully defined AI by contrasting it with the status quo: Theres a significant difference between AI, which to me denotes machine learning and machines moving forward on their own, versus auto-decisioning, which is using data within the context of a managed decision algorithm.

Americas current legal and regulatory structure to protect against discrimination and enforce fair lending is not well equipped to handle AI. The foundation is a set of laws from the 1960s and 1970s (Equal Credit Opportunity Act of 1974, Truth in Lending Act of 1968, Fair Housing Act of 1968, etc.) that were based on a time with almost the exact opposite problems we face today: not enough sources of standardized information to base decisions and too little credit being made available. Those conditions allowed rampant discrimination by loan officers who could simply deny people because they didnt look credit worthy.

Today, we face an overabundance of poor-quality credit (high interest rates, fees, abusive debt traps) and concerns over the usage of too many sources of data that can hide as proxies for illegal discrimination. The law makes it illegal to use gender to determine credit eligibility or pricing, but countless proxies for gender exist from the type of deodorant you buy to the movies you watch.

Americas current legal and regulatory structure to protect against discrimination and enforce fair lending is not well equipped to handle AI.

The key concept used to police discrimination is that of disparate impact. For a deep dive into how disparate impact works with AI, you can read my previous work on this topic. For this article, it is important to know that disparate impact is defined by the Consumer Financial Protection Bureau as when: A creditor employs facially neutral policies or practices that have an adverse effect or impact on a member of a protected class unless it meets a legitimate business need that cannot reasonably be achieved by means that are less disparate in their impact.

The second half of the definition provides lenders the ability to use metrics that may have correlations with protected class elements so long as it meets a legitimate business need,andthere are no other ways to meet that interest that have less disparate impact. A set of existing metrics, including income, credit scores (FICO), and data used by the credit reporting bureaus, has been deemed acceptable despite having substantial correlation with race, gender, and other protected classes.

For example, consider how deeply correlated existing FICO credit scores are with race. To start, it is telling how little data is made publicly available on how these scores vary by race. The credit bureau Experian is eager to publicize one of its versions of FICO scores by peoples age, income, and even what state or city they live in, but not by race. However, federal law requires lenders to collect data on race for home mortgage applications, so we do have access to some data. As shown in the figure below, the differences are stark.

Among people trying to buy a home, generally a wealthier and older subset of Americans, white homebuyers have an average credit score 57 points higher than Black homebuyers and 33 points higher than Hispanic homebuyers. The distribution of credit scores is also sharply unequal: More than 1 in 5 Black individuals have FICOs below 620, as do 1 in 9 among the Hispanic community, while the same is true for only 1 out of every 19 white people. Higher credit scores allow borrowers to access different types of loans and at lower interest rates. One suspects the gaps are even broader beyond those trying to buy a home.

If FICO were invented today, would it satisfy a disparate impact test? The conclusion of Rice and Swesnik in their law review article was clear: Our current credit-scoring systems have a disparate impact on people and communities of color. The question is mute because not only is FICO grandfathered, but it has also become one of the most important factors used by the financial ecosystem. I have described FICO as the out of tune oboe to which the rest of the financial orchestra tunes.

New data and algorithms are not grandfathered and are subject to the disparate impact test. The result is a double standard whereby new technology is often held to a higher standard to prevent bias than existing methods. This has the effect of tilting the field against new data and methodologies, reinforcing the existing system.

Explainability is another core tenant of our existing fair lending system that may work against AI adoption. Lenders are required to tell consumers why they were denied. Explaining the rationale provides a paper trail to hold lenders accountable should they be engaging in discrimination. It also provides the consumer with information to allow them to correct their behavior and improve their chances for credit. However, an AIs method to make decisions may lack explainability. As Federal Reserve Governor Lael Brainard described the problem: Depending on what algorithms are used, it is possible that no one, including the algorithms creators, can easily explain why the model generated the results that it did. To move forward and unlock AIs potential, we need a new conceptual framework.

To start, imagine a trade-off between accuracy (represented on the y-axis) and bias (represented on the x-axis). The first key insight is that the current system exists at the intersection of the axes we are trading off: the graphs origin. Any potential change needs to be considered against the status-quonot an ideal world of no bias nor complete accuracy. This forces policymakers to consider whether the adoption of a new system that contains bias, but less than that in the current system, is an advance. It may be difficult to embrace an inherently biased framework, but it is important to acknowledge that the status quo is already highly biased. Thus, rejecting new technology because it contains some level of bias does not mean we are protecting the system against bias. To the contrary, it may mean that we are allowing a more biased system to perpetuate.

As shown in the figure above, the bottom left corner (quadrant III) is one where AI results in a system that is more discriminatory and less predictive. Regulation and commercial incentives should work together against this outcome. It may be difficult to imagine incorporating new technology that reduces accuracy, but it is not inconceivable, particularly given the incentives in industry to prioritize decision-making and loan generation speed over actual loan performance (as in the subprime mortgage crisis). Another potential occurrence of policy moving in this direction is the introduction of inaccurate data that may confuse an AI into thinking it has increased accuracy when it has not. The existing credit reporting system is rife with errors: 1 out of every 5 people may have material error on their credit report. New errors occur frequentlyconsider the recent mistake by one student loan servicer that incorrectly reported 4.8 million Americans as being late on paying their student loans when in fact in the government had suspended payments as part of COVID-19 relief.

The data used in the real world are not as pure as those model testing. Market incentives alone are not enough to produce perfect accuracy; they can even promote inaccuracy given the cost of correcting data and demand for speed and quantity. As one study from the Federal Reserve Bank of St. Louis found, Credit score has not acted as a predictor of either true risk of default of subprime mortgage loans or of the subprime mortgage crisis. Whatever the cause, regulators, industry, and consumer advocates ought to be aligned against the adoption of AI that moves in this direction.

The top right (quadrant I) represents incorporation of AI that increases accuracy and reduces bias. At first glance, this should be a win-win. Industry allocates credit in a more accurate manner, increasing efficiency. Consumers enjoy increased credit availability on more accurate terms and with less bias than the existing status quo. This optimistic scenario is quite possible given that a significant source of existing bias in lending stems from the information used. As the Bank Policy Institute pointed out in its in discussion draft of the promises of AI: This increased accuracy will benefit borrowers who currently face obstacles obtaining low-cost bank credit under conventional underwriting approaches.

One prominent example of a win-win system is the use of cash-flow underwriting. This new form of underwriting uses an applicants actual bank balance over some time frame (often one year) as opposed to current FICO based model which relies heavily on seeing whether a person had credit in the past and if so, whether they were ever in delinquency or default. Preliminary analysis by FinReg Labs shows this underwriting system outperforms traditional FICO on its own, and when combined with FICO is even more predictive.

Cash-flow analysis does have some level of bias as income and wealth are correlated with race, gender, and other protected classes. However, because income and wealth are acceptable existing factors, the current fair-lending system should have little problem allowing a smarter use of that information. Ironically, this new technology meets the test because it uses data that is already grandfathered.

That is not the case for other AI advancements. New AI may increase credit access on more affordable terms than what the current system provides and still not be allowable. Just because AI has produced a system that is less discriminatory does not mean it passes fair lending rules. There is no legal standard that allows for illegal discrimination in lending because it is less biased than prior discriminatory practices. As a 2016 Treasury Department study concluded, Data-driven algorithms may expedite credit assessments and reduce costs, they also carry the risk of disparate impact in credit outcomes and the potential for fair lending violations.

For example, consider an AI that is able, with a good degree of accuracy, to detect a decline in a persons health, say through spending patterns (doctors co-pays), internet searches (cancer treatment), and joining new Facebook groups (living with cancer). Medical problems are a strong indicator of future financial distress. Do we want a society where if you get sick, or if a computer algorithm thinks you are ill, that your terms of credit decrease? That may be a less biased system than we currently have, and not one that policymakers and the public would support. Of all sudden what seems like a win-win may not actually be one that is so desirable.

AI that increases accuracy but introduces more bias gets a lot of attention, deservedly so. This scenario represented in the top left (quadrant II) of this framework can range from the introduction of data that are clear proxies for protected classes (watch Lifetime or BET on TV) to information or techniques that, on a first glance, do not seem biased but actually are. There are strong reasons to believe that AI will naturally find proxies for race, given that there are large income and wealth gaps between races. As Daniel Schwartz put it in his article on AI and proxy discrimination: Unintentional proxy discrimination by AIs is virtually inevitable whenever the law seeks to prohibit discrimination on the basis of traits containing predictive information that cannot be captured more directly within the model by non-suspect data.

Proxy discrimination by AI is even more concerning because the machines are likely to uncover proxies that people had not previously considered.

Proxy discrimination by AI is even more concerning because the machines are likely to uncover proxies that people had not previously considered. Think about the potential to use whether or not a person uses a Mac or PC, a factor that is both correlated to race and whether people pay back loans, even controlling for race.

Duke Professor Manju Puri and co-authors were able to build a model using non-standard data that found substantial predictive power in whether a loan was repaid through whether that persons email address contained their name. Initially, that may seem like a non-discriminatory variable within a persons control. However, economists Marianne Bertrand and Sendhil Mullainathan have shown African Americans with names heavily associated with their race face substantial discrimination compared to using race-blind identification. Hence, it is quite possible that there is a disparate impact in using what seems like an innocuous variable such as whether your name is part of your email address.

The question for policymakers is how much to prioritize accuracy at a cost of bias against protected classes. As a matter of principle, I would argue that our starting point is a heavily biased system, and we should not tolerate the introduction of increased bias. There is a slippery slope argument of whether an AI produced substantial increases in accuracy with the introduction of only slightly more bias. Afterall, our current system does a surprisingly poor job of allocating many basic credits and tolerates a substantially large amount of bias.

Industry is likely to advocate for inclusion of this type of AI while consumer advocates are likely to oppose its introduction. Current law is inconsistent in its application. Certain groups of people are afforded strong anti-discrimination protection against certain financial products. But again, this varies across financial product. Take gender for example. It is blatantly illegal under fair lending laws to use gender or any proxy for gender in allocating credit. However, gender is a permitted use for price difference for auto insurance in most states. In fact, for brand new drivers, gender may be the single biggest factor used in determining price absent any driving record. America lacks a uniform set of rules on what constitutes discrimination and what types of attributes cannot be discriminated against. Lack of uniformity is compounded by the division of responsibility between federal and state governments and, within government, between the regulatory and judicial system for detecting and punishing crime.

The final set of trade-offs involve increases in fairness but reductions in accuracy (quadrant IV in the bottom right). An example includes an AI with the ability to use information about a persons human genome to determine their risk of cancer. This type of genetic profiling would improve accuracy in pricing types of insurance but violates norms of fairness. In this instance, policymakers decided that the use of that information is not acceptable and have made it illegal. Returning to the role of gender, some states have restricted the use of gender in car insurance. California most recently joined the list of states no longer allowing gender, which means that pricing will be more fair but possibly less accurate.

Industry pressures would tend to fight against these types of restrictions and press for greater accuracy. Societal norms of fairness may demand trade-offs that diminish accuracy to protect against bias. These trade-offs are best handled by policymakers before the widespread introduction of this information such as the case with genetic data. Restricting the use of this information, however, does not make the problem go away. To the contrary, AIs ability to uncover hidden proxies for that data may exacerbate problems where society attempts to restrict data usage on the grounds of equity concerns. Problems that appear solved by prohibitions then simply migrate into the algorithmic world where they reappear.

The underlying takeaway for this quadrant is one in which social movements that expand protection and reduce discrimination are likely to become more difficult as AIs find workarounds. As long as there are substantial differences in observed outcomes, machines will uncover differing outcomes using new sets of variables that may contain new information or may simply be statistically effective proxies for protected classes.

The status quo is not something society should uphold as nirvana. Our current financial system suffers not only from centuries of bias, but also from systems that are themselves not nearly as predictive as often claimed. The data explosion coupled with the significant growth in ML and AI offers tremendous opportunity to rectify substantial problems in the current system. Existing anti-discrimination frameworks are ill-suited to this opportunity. Refusing to hold new technology to a higher standard than the status quo results in an unstated deference to the already-biased current system. However, simply opening the flood gates under the rules of can you do better than today opens up a Pandoras box of new problems.

The status quo is not something society should uphold as nirvana. Our current financial system suffers not only from centuries of bias, but also from systems that are themselves not nearly as predictive as often claimed.

Americas fractured regulatory system, with differing roles and responsibilities across financial products and levels of government, only serves to make difficult problems even harder. With lacking uniform rules and coherent frameworks, technological adoption will likely be slower among existing entities setting up even greater opportunities for new entrants. A broader conversation regarding how much bias we are willing to tolerate for the sake of improvement over the status quo would benefit all parties. That requires the creation of more political space for sides to engage in a difficult and honest conversation. The current political moment in time is ill-suited for that conversation, but I suspect that AI advancements will not be willing to wait until America is more ready to confront these problems.

The Brookings Institution is a nonprofit organization devoted to independent research and policy solutions. Its mission is to conduct high-quality, independent research and, based on that research, to provide innovative, practical recommendations for policymakers and the public. The conclusions and recommendations of any Brookings publication are solely those of its author(s), and do not reflect the views of the Institution, its management, or its other scholars.

Microsoft provides support to The Brookings InstitutionsArtificial Intelligence and Emerging Technology (AIET) Initiative, and Apple, Facebook, and IBM provide general, unrestricted support to the Institution. The findings, interpretations, and conclusions in this report are not influenced by any donation. Brookings recognizes that the value it provides is in its absolute commitment to quality, independence, and impact. Activities supported by its donors reflect this commitment.

Go here to see the original:

Reducing bias in AI-based financial services - Brookings Institution