AI artificial intelligence concept Central Computer Processors CPU concept, 3d rendering, Circuit ... [+] board, Technology background, Motherboard digital chip, Tech science background, machine learning

Machine learning is definitely a confusing term.Is it AI or something different?

Well, its actually a subset of AI (which, by the way, is a massive category). Machine learning is a method of analyzing data using an analytical model that is built automatically, or learned, from training data, said Rick Negrin, who is the VP of Product Management at MemSQL. The idea is that the model gets better as you feed it more data points.

There are two key steps with machine learning.First, you need to collect and train the data, which can be a long and tough process.Then, you will operationalize the machine learning, such as by using it to help provide insights or as part of a product.There are a myriad of tools to help with the process, such as open source platforms like TensorFlow and commercial systems, such as DataRobot.

Successful machine learning is only as good as the data available, which is why it needs new, updated data to provide the most accurate outputs or predictions for any given need, said Panagiotis Angelopoulos, who is the Chief Data Officer at Persado.And unlike what any one person can analyze, machine learning can take vast amounts of data over time and make predictions to improve the customer experience and provide real value to the end-user.

Sometimes the models are so intricate that they are nearly impossible to understand. The lack of transparency can make it so that certain industries, like healthcare and banking, may not be able to use machine learning models. Because of this, more research is being focused on the explainability of models.

Another challenge with machine learning is the need to form an experienced team. To build this team in-house, you will have to hire more than just data scientists, said Ji Li, who is the director of data science at CLARA analytics.Full deployment of a new solution requires product managers, software engineers, data engineers, operational experts to develop process and operational workflows, staff to integrate data models into operations, people to manage onboarding and training of the employees who will ultimately use the solution, and staff who can quantify value generation.

In other words, for many organizations, the best option with machine learning may be to buy an off-the-shelf solution.The good news is that there are many on the marketand they are generally affordable.

But regardless of what path you take, there needs to be a clear-cut business case for machine learning.It should not be used just because it is trendy.There also needs to be sufficient change management within the organization. One of the greatest challenges in implementing machine learning and other data science initiatives is navigating institutional changegetting a buy-in, dealing with new processes, the changing job duties, and more, said Ingo Mierswa, who is the founder and CTO of RapidMiner.

Then what are the use cases for machine learning?According to Alyssa Simpson Rochwerger, who is the VP of AI and the Data Evangelist at Appen:Machine learning can solve lots of different types of problems.But it's particularly well suited to decisions that require very simple and repetitive tasks at large scale. For example, the US Postal Service has been successfully using machine learning systems to sort the mail for decades. The task was simple:read the address on the mail (sense) and then understand the zip code (perceive) and then sort into different buckets (decide). The US Postal Service processes almost two hundred million pieces of mail per dayso sorting this by hand wouldn't work.

In fact, the examples are seemingly endless for machine learning.Here are just a few:

Machine learning is a tool and like most tools, it works best when used properly, said Matei Zaharia, who is the chief technologist and co-founder of Databricks.Machine learning can take something as simple as some images and some annotations or just drawings on those images and create a solution that can be automated efficiently and at scale. However, we are not in a technological state where a machine learning model can just work on anything that is thrown at itthat is, not without some kind of external guidance. A machine learns, a human teaches.

Tom (@ttaulli) is an advisor to startups and the author of Artificial Intelligence Basics: A Non-Technical Introduction and The Robotic Process Automation Handbook: A Guide to Implementing RPA Systems. He also has developed various online courses, such as for the Python programming language.

Read the original here:
Machine Learning: What Is It Really Good For? - Forbes

Machine-learning algorithms are responsible for the vast majority of the artificial intelligence advancements and applications you hear about. (For more background, check out our first flowchart on "What is AI?" here.)

Machine-learning algorithms use statistics to find patterns in massive* amounts of data. And data, here, encompasses a lot of thingsnumbers, words, images, clicks, what have you. If it can be digitally stored, it can be fed into a machine-learning algorithm.

Machine learning is the process that powers many of the services we use todayrecommendation systems like thoseon Netflix, YouTube, and Spotify; search engines like Google and Baidu; social-media feeds like Facebook and Twitter; voice assistants like Siri and Alexa. The list goes on.

In all of these instances, each platform is collecting as much data about you as possiblewhat genres you like watching, what links you are clicking, which statuses you are reacting toand using machine learning to make a highly educated guess about what you might want next. Or, in the case of a voice assistant, about which words match best with the funny sounds coming out of your mouth.

Frankly, this process is quite basic: find the pattern, apply the pattern. But it pretty much runs the world. Thats in big part thanks to an invention in 1986, courtesy of Geoffrey Hinton, today known as the father of deep learning.

Deep learning is machine learning on steroids: it uses a technique that gives machines an enhanced ability to findand amplifyeven the smallest patterns. This technique is called a deep neural networkdeep because it has many, many layers of simple computational nodes that work together to munch through data and deliver a final resultin the form of the prediction.

Neural networks were vaguely inspired by the inner workings of the human brain. The nodes are sort of like neurons, and the network is sort of like the brain itself. (For the researchers among you who are cringing at this comparison: Stop pooh-poohing the analogy. Its a good analogy.) But Hinton published his breakthrough paper at a time when neural nets had fallen out of fashion. No one really knew how to train them, so they werent producing good results. It took nearly 30 years for the technique to make a comeback. And boy, did it make a comeback.

One last thing you need to know: machine (and deep) learning comes in three flavors: supervised, unsupervised, and reinforcement. In supervised learning, the most prevalent, the data is labeled to tell the machine exactly what patterns it should look for. Think of it as something like a sniffer dog that will hunt down targets once it knows the scent its after. Thats what youre doing when you press play on a Netflix showyoure telling the algorithm to find similar shows.

In unsupervised learning, the data has no labels. The machine just looks for whatever patterns it can find. This is like letting a dog smell tons of different objects and sorting them into groups with similar smells. Unsupervised techniques arent as popular because they have less obvious applications. Interestingly, they have gained traction incybersecurity.

Lastly, we have reinforcement learning, the latest frontier of machine learning. A reinforcement algorithm learns by trial and error to achieve a clear objective. It tries out lots of different things and is rewarded or penalized depending on whether its behaviors help or hinder it from reaching its objective. This is like giving and withholding treats when teaching a dog a new trick. Reinforcement learning is the basis of Googles AlphaGo, the program that famously beat the best human players in the complex game of Go.

Thats it. That's machine learning. Now check out the flowchart above for a final recap.

*Note: Okay, there are technically ways to perform machine learning on smallish amounts of data, but you typically need huge piles of it to achieve good results.

___

This originally appeared in our AI newsletter The Algorithm. To have it directly delivered to your inbox,subscribe herefor free.

More here:
What is machine learning? | MIT Technology Review

A newly released tool makes light work of solving human verification challenges

F-Secure says its achieved 90% accuracy in cracking Microsoft Outlooks text-based CAPTCHAs using its AI-based CAPTCHA-cracking server, CAPTCHA22.

For the last two years, the security firm has been using machine learning techniques to train unique models that solve a particular CAPTCHA, rather than trying to build a one-size-fits-all model.

And, recently, it decided to try the system out on a CAPTCHA used by an Outlook Web App (OWA) portal.

The initial attempt, according to F-Secure, was comparatively unsuccessful, with the team finding that after manually labelling around 200 CAPTCHAs, it could only identify the characters with an accuracy of 22%.

The first issue to emerge was noise, with the team determining that the greyscale value of noise and text was always within two distinct and constant ranges. Tweaks to the tool helped filter out the noise.

The team also realized that some of the test CAPTCHAs had been labelled incorrectly, with confusion between, for example, l and I (lower case L and upper case i). Fixing this shortcoming brought the accuracy up to 47%.

More challenging, though, was handling the CAPTCHA submission to Outlooks web portal.

There was no CAPTCHA POST request, with the CAPTCHA instead sent as a value appended to a cookie. JavaScript was used to keylog the user as the answer to the CAPTCHA was typed.

Instead of trying to replicate what occurred in JS, we decided to use Pyppeteer, a browsing simulation Python package, to simulate a user entering the CAPTCHA, said Tinus Green, a senior information security consultant at F-Secure

Doing this, the JS would automatically take care of the submission for us.

Green added: We could use this simulation software to solve the CAPTCHA whenever it blocked entries and once solved, we could continue with our conventional attack, hence automating the process once again.

We have now also refactored CAPTCHA22 for a public release.

CAPTCHAs are challenge-response tests used by many websites in an attempt to distinguish between genuine requests to sign-up to or access web services by a human user and automated requests by bots.

Spammers, for example, attempt to circumvent CAPTCHAs in order to create accounts they can later abuse to distribute junk mail.

CAPTCHAs are something of a magnet for cybercriminals and security researchers, with web admins struggling to stay one step ahead.

Late last year, for example, PortSwigger Web Security uncovered a security weakness in Googles reCAPTCHA that allowed it to be partially bypassed by using Turbo Intruder, a research-focused Burp Suite extension, to trigger a race condition.

Soon after, a team of academics from the University of Maryland was able to circumvent Googles reCAPTCHA v2s anti-bot mechanism using a Python-based program called UnCaptcha, which could solve its audio challenges.

Green said: There is a catch 22 between creating a CAPTCHA that is user friendly grandma safe as we call it and sufficiently complex to prevent solving through computers. At this point it seems as if the balance does not exist.

Web admins shouldnt, he says, give away half the required information through username enumeration, and users should be required to set strong pass phrases conforming to NIST standards.

And, he adds: Accept that accounts can be breached, and therefore implement MFA [multi-factor authentication] as an additional barrier.

RELATED New tool highlights shortcomings in reCAPTCHAs anti-bot engine

Read the original post:
Machine learning techniques applied to crack CAPTCHAs - The Daily Swig

TEL AVIV, Israel, May 26, 2020 /PRNewswire/ --cnvrg.io, the data science platform simplifying model management and introducing advanced MLOps to the industry, today announced its streaming endpoints solution, a new capability for deploying ML models to production with Apache Kafka in one click. cnvrg.iois the first ML platform to enable one click streaming endpoint deployment for large-scale and real-time predictions with high throughput and low latency.

85% of machine learning models don't get to production due to the technical complexity of deploying the model in the right environment and architecture. Models can be deployed in a variety of different ways. Batch deployment for offline inference and web service for more real-time scenarios. These two approaches cover most of the ML use cases, but they both fall short in an enterprise setting when you need to scale and stream millions of predictions in real time. Enterprises require fast, scalable predictions to execute critical and time sensitive business decisions.

cnvrg.iois thrilled to announce its new capability of deploying ML models to production with a streaming architecture of producer/consumer interface with native integration to Apache Kafka and AWS Kinesis. In just one click, data scientists and engineers can deploy any kind of model as an endpoint that can receive data as stream and output predictions as streams.

Deployed models will be tracked with advanced model management and model monitoring solutions including alerts, retraining, A/B testing and canary rollout, autoscaling and more.

This new capability allows engineers to support and predict millions of samples in a real-time environment. This architecture is ideal for time sensitive or event-based predictions, recommender systems, and large-scale applications that require high throughput, low latency and fault tolerant environments.

"Playtika has 10 million daily active users (DAU), 10 billion daily events and over 9TB of daily processed data for our online games. To provide our players with a personalized experience, we need to ensure our models run at peak performance at all times," says Avi Gabay, Director of Architecture at Playtika. "With cnvrg.io we were able to increase our model throughput by up to 50% and on average by 30% when comparing to RESTful APIs. cnvrg.io also allows us to monitor our models in production, set alerts and retrain with high-level automation ML pipelines."

The new cnvrg.io release extends the market footprint and enhances the prior announcements of NVIDIA DGX-Ready partnership and Red Hat unified control plane.

About cnvrg.io

cnvrg.io is an AI OS, transforming the way enterprises manage, scale and accelerate AI and data science development from research to production. The code-first platform is built by data scientists, for data scientists and offers unrivaled flexibility to run on-premise or cloud.

Logo - https://mma.prnewswire.com/media/1160338/cnvrg_io_Logo.jpg

SOURCE cnvrg.io

Full Stack Machine Learning Operating System

Go here to see the original:
cnvrg.io Releases New Streaming Endpoints With One-click Deployment for Real-time Machine Learning Applications - PRNewswire

The mobile computing trend requires enterprises to meet consumers' expectations for accessing information and completing tasks from a smartphone. But there's a converse to that arrangement: Mobile has also become the go-to digital platform companies use to market their goods and services.

Align Technology, which offers the Invisalign orthodontic device to straighten teeth, is embracing the trend with a mobile platform that both helps patients coordinate care with their doctors and entices new customers. The My Invisalign app includes detailed content on how the Invisalign system works, as well as machine learning (ML) technology to simulate what wearers' smiles will look like after using the medical device.

"It's a natural extension to help doctors and patients stay in touch," says Align Technology Chief Digital Officer Sreelakshmi Kolli, who joined the company as a software engineer in 2003 and has spent the past few years digitizing the customer experience and business operations. The development of My Invisalign also served as a pivot point for Kolli to migrate the company to agile and DevSecOps practices.

My Invisalign is a digital on-ramp for a company that has relied on pitches from enthusiastic dentists and pleased patients to help Invisalign find a home in the mouths of more than 8 million customers. An alternative to clunky metal braces, Invisalign comprises sheer plastic aligners that straighten patients' teeth gradually over several months. Invisalign patients swear by the device, but many consumers remain on the fence about a device with a $3,000 to $5,000 price range that is rarely covered completely by insurance.

Visit link:
Machine learning helps Invisalign patients find their perfect smile - CIO

DUBLIN--(BUSINESS WIRE)--The "Global Mining Ventilation Systems Market 2020-2024" report has been added to ResearchAndMarkets.com's offering.

The mining ventilation systems market is poised to grow by $ 81.73 mn during 2020-2024 progressing at a CAGR of 4% during the forecast period. This report on the mining ventilation systems market provides a holistic analysis, market size and forecast, trends, growth drivers, and challenges, as well as key vendor analysis.

The market is driven by the growing demand for safety in underground mining and demand for minerals. In addition, increasing demand for precious metals is anticipated to boost the growth of the market as well. This study identifies technological advances as one of the prime reasons driving the mining ventilation systems market growth during the next few years. Also, the growing adoption of AI and machine learning and increasing use of drones will lead to sizable demand in the market.

The mining ventilation systems market analysis includes product segment and geographic landscapes

The mining ventilation systems market covers the following areas:

This robust vendor analysis is designed to help clients improve their market position, and in line with this, this report provides a detailed analysis of several leading mining ventilation systems market vendors that include ABB Ltd., ABC Canada Technology Group Ltd., ABC Industries Inc., Epiroc AB, Howden Group Ltd., New York Blower Co., Sibenergomash-BKZ LLC, Stantec Inc., TLT-Turbo GmbH, and Zitron SA. Also, the mining ventilation systems market analysis report includes information on upcoming trends and challenges that will influence market growth. This is to help companies strategize and leverage on all forthcoming growth opportunities.

The study was conducted using an objective combination of primary and secondary information including inputs from key participants in the industry. The report contains a comprehensive market and vendor landscape in addition to an analysis of the key vendors.

This study presents a detailed picture of the market by the way of study, synthesis, and summation of data from multiple sources by an analysis of key parameters such as profit, pricing, competition, and promotions. It presents various market facets by identifying the key industry influencers. The data presented is comprehensive, reliable, and a result of extensive research - both primary and secondary.

The market research report provide a complete competitive landscape and an in-depth vendor selection methodology and analysis using qualitative and quantitative research to forecast an accurate market growth.

Key Topics Covered:

Executive Summary

Market Landscape

Market Sizing

Five Forces Analysis

Market Segmentation by Product

Customer landscape

Geographic Landscape

Vendor Landscape

Vendor Analysis

Companies Mentioned

For more information about this report visit https://www.researchandmarkets.com/r/yl3xpg

About ResearchAndMarkets.com

ResearchAndMarkets.com is the world's leading source for international market research reports and market data. We provide you with the latest data on international and regional markets, key industries, the top companies, new products and the latest trends.

Read more from the original source:
Growing Adoption of AI and Machine Learning and Increased Use of Drones is Driving Growth in the Global Mining Ventilation Systems Market -...

As part of its commitment to using data and analytics to solve the world's most pressing problems, SAS' recent work includes helping to save the world's No. 1 food crop pollinator the honey bee. With the number of bee colonies drastically declining around the world, SAS is using technology such as theInternet of Things (IoT), machine learning and visual analytics to help maintain and support healthy bee populations.

In honor of World Bee Day, SAS is highlighting three separate projects where technology is monitoring, tracking and improving pollinator populations around the globe. First, researchers at SAS have developed a noninvasive way to monitor real-time conditions of beehives through auditory data and machine learning algorithms. SAS is also working withAppalachian State Universityon the World Bee Count to visualize world bee population data and understand the best ways to save them. Lastly, recent SASViyaHackathon winners decoded bee communication through machine learning in order to maximize their food access and boost human food supplies.

"SAS has always looked for ways to use technology for a better world," said Oliver Schabenberger, COO and CTO of SAS. "By applying advanced analytics and artificial intelligence to beehive health, we have a better shot as a society to secure this critically important part of our ecosystem and, ultimately, our food supply."

Noninvasively Monitoring Beehive HealthResearchers from the SAS IoT Division are developing abioacoustic monitoring systemto noninvasively track real-time conditions of beehives using digital signal processing tools and machine learning algorithms available in SASEvent Stream Processingand SAS Viya software. This system helps beekeepers better understand and predict hive problems which could lead to colony failure, including the emergence of new queens something they would not ordinarily be able to detect.

Annual loss rates of U.S. beehives exceed 40%, and between 25% and 40% of these losses are due to queen failure. Acoustic analysis can alert beekeepers to queen disappearances immediately, which is vitally important to significantly reducing colony loss rates. With this system, beekeepers will have a deeper understanding of their hives without having to conduct time-consuming and disruptive manual inspections.

"As a beekeeper myself, I know the magnitude of bees' impact on our ecosystem, and I'm inspired to find innovative ways to raise healthier bees to benefit us all," saidAnya McGuirk, Distinguished Research Statistician Developer in the IoT division at SAS. "And as a SAS employee, I'm proud to have conducted this experiment with SAS software at our very own campus beehives, demonstrating both the power of our analytical capabilities and our commitment to innovation and sustainability."

By connecting sensors to SAS' four Bee Downtown hives at its headquarters inCary, NC, the team startedstreaming hive datadirectly to the cloud to continuously measure data points in and around the hive, including weight, temperature, humidity, flight activity and acoustics. In-stream machine learning models were used to "listen" to the hive sounds, which can indicate health, stress levels, swarming activities and the status of the queen bee. To ensure only the hum of the hive was being used to determine bees' health and happiness, researchers used robust principal component analysis (RPCA), a machine learning technique, to separate extraneous or irrelevant noises from the inventory of sounds collected by hive microphones.

The researchers found that with RPCA capabilities, they could detect worker bees "piping" at the same frequency range at which a virgin queen pipes after a swarm, likely to assess whether a queen was present. The researchers then designed an automated pipeline to detect either queen piping following a swarm or worker piping that occurs when the colony is queenless. This is greatly beneficial to beekeepers, warning them that a new queen may be emerging and giving them the opportunity to intervene before significant loss occurs.

The researchers plan to implement the acoustic streaming system very soon and are continuing to look for ways to broaden the usage of technology to help honey bees and ultimately humankind.

Visualizing the World's Pollinator PopulationsOn World Bee Day, SAS is launching a data visualization that maps out bees "counted" around the globe for theWorld Bee Count, an initiative co-founded by theCenter for Analytics Research and Education(CARE) atAppalachian State University. The goal of a World Bee Count is to engage citizens across the world to take pictures of bees as a first step toward understanding the reasons for their alarming decline.

"The World Bee Count allows us to crowdsource bee data to both visualize our planet's bee population and create one of the largest, most informative data sets about bees to date," saidJoseph Cazier, Professor and Executive Director atAppalachian State University'sCARE. "SAS' data visualization will show the crowdsourced location of bees and other pollinators. In a later phase of the project, researchers can overlay key data points like crop yield, precipitation and other contributing factors of bee health, gathering a more comprehensive understanding of our world's pollinators." Bayer has agreed to help sponsor CARE to allow its students and faculty to perform research on the World Bee Count data and other digital pollinator data sources.

In early May, the World Bee Count app was launched for users both beekeepers and the general public, aka "citizen data scientists" to add data points to the Global Pollinator Map. Within the app, beekeepers can enter the number of hives they have, and any user can submit pictures of pollinators from their camera roll or through the in-app camera. Through SAS Visual Analytics, SAS has created avisualization mapto display the images users submit via the app. In addition to showing the results of the project, the visualizations can potentially provide insights about the conditions that lead to the healthiest bee populations.

In future stages of this project, the robust data set created from the app could help groups like universities and research institutes better strategize ways to save these vital creatures.

Using Machine Learning to Maximize Bees' Access to FoodRepresenting the Nordic region, a team from Amesto NextBridgewon the 2020 SAS EMEA Hackathon, which challenged participants to improve sustainability using SAS Viya. Their winning project used machine learning to maximize bees' access to food, which would in turn benefit mankind's food supply. In partnership withBeefutures, the team successfully accomplished this by developing a system capable of automatically detecting, decoding and mapping bee "waggle" dances using Beefutures' observation hives and SAS Viya.

Bees are responsible for pollinating nearly 75% of all plant species directly used for human food, but the number of bee colonies are declining, which will lead to a devastating loss for human food supply. A main reason for the decline of bee populations is a lack of access to food due to an increase in monoculture farming. When bees do find a good food source, they come back to the hive to communicate its exact location through a "waggle dance." By observing these dances, beekeepers can better understand where their bees are getting food and then consider establishing new hives in these locations to help maintain strong colonies.

"Observing all of these dances manually is virtually impossible, but by using video footage from inside the hives and training machine learning algorithms to decode the dance, we will be able to better understand where bees are finding food," said Kjetil Kalager, lead of the Amesto NextBridge and Beefutures team. "We implemented this information, along with hive coordinates, sun angle, time of day and agriculture around the hives into an interactive map in SAS Viya and then beekeepers can easily decode this hive information and relocate to better suited environments if necessary."

This systematic real-time monitoring of waggle dances allows bees to act as sensors for their ecosystems. Further research using this technology may uncover other information bees communicate through dance that could help us save and protect their population, which ultimately benefits us all.

See thiswaggle dance project in actionand learn about howSAS is committed to corporate social responsibility.

Read more from the original source:
Advanced Analytics and Machine Learning Boost Bee Populations - Transmission & Distribution World

In the world of SEO, its important to understand the system youre optimizing for.

You need to understand how:

Another crucial area to understand is machine learning.

Now, the term machine learning gets thrown around a lot these days.

But how does machine learning actually impact search and SEO?

This chapter will explore everything you need to know about how search engines use machine learning.

It would be difficult to understand how search engines use machine learning without knowing what machine learning actually is.

ADVERTISEMENT

CONTINUE READING BELOW

Lets start with the definition (provided by Stanford University in their course description for Coursera) before we move on to a practical explanation:

Machine learning is the science of getting computers to act without being explicitly programmed.

Machine learning isnt the same as Artificial Intelligence (AI), but the line is starting to get a bit blurry with the applications.

As noted above, machine learning is the science of getting computers to come to conclusions based on information but without being specifically programmed in how to accomplish said task.

AI, on the other hand, is the science behind creating systems that either have, or appear to possess, human-like intelligence and process information in a similar manner.

Think of the difference this way:

Machine learning is a system designed to solve a problem. It works mathematically to produce the solution.

The solution could be programmed specifically, or worked out by humans manually, but without this need, the solutions come much faster.

ADVERTISEMENT

CONTINUE READING BELOW

A good example would be setting a machine off to pour through oodles of data outlining tumor size and location without programming in what its looking for. The machine would be given a list of known benign and malignant conclusions.

With this, we would then ask the system to produce a predictive model for future encounters with tumors to generate odds in advance as to which it is based on the data analyzed.

This is purely mathematical.

A few hundred mathematicians could do this but it would take them many years (assuming a very large database) and hopefully, none of them would make any errors.

Or, this same task could be accomplished with machine learning in far less time.

When Im thinking of Artificial Intelligence, on the other hand, thats when I start to think of a system that touches on the creative and thus becomes less predictable.

An artificial intelligence set on the same task may simply reference documents on the subject and pull conclusions from previous studies.

Or it may add new data into the mix.

Or may start working on a new system of electrical engine, foregoing the initial task.

It probably wont get distracted on Facebook, but you get where Im going.

The key word is intelligence.

While artificial, to meet the criteria it would have to be real thus producing variables and unknowns akin to what we encounter when we interact with others around us.

Right now what the search engines (and most scientists) are pushing to evolve is machine learning.

Google has a freecourse on it, has made its machine learning frameworkTensorFlow open source, and is makingbig investments in hardware to run it.

Basically, this is the future so its best to understand it.

While we cant possibly list (or even know) every application of machine learning going on over at the Googleplex, lets look at a couple of known examples:

ADVERTISEMENT

CONTINUE READING BELOW

What article on machine learning at Google would be complete without mentioning their first and still highly-relevant implementation of a machine learning algorithm into search?

Thats right were talkingRankBrain.

Essentially the system was armed only with an understanding of entities (a thing or concept that is singular, unique, well-defined, and distinguishable) and tasked with producing an understanding of how those entities connect in a query to assist in better understanding the query and a set of known good answers.

These are brutally simplified explanations of both entities and RankBrain but it serves our purposes here.

So, Google gave the system some data (queries) and likely a set of known entities.

Im going to guess on the next process but logically the system would then be tasked with training itself based on the seed set of entities on how to recognize unknown entities it encounters.

The system would be pretty useless if it wasnt able to understand a new movie name, date, etc.

ADVERTISEMENT

CONTINUE READING BELOW

Once the system had that process down and was producing satisfactory results they would have then tasked it with teaching itself how to understand the relationships between entities and what data is being implied or directly requested and seek out appropriate results in the index.

This system solves many problems that plagued Google.

The requirement to include keywords like How do I replace my S7 screen on a page about replacing one should not be necessary.

You also shouldnt have to include fix if youve included replace as, in this context, they generally imply the same thing.

RankBrain uses machine learning to:

In its first iteration, RankBrain was tested on queries Google had not encountered before. This makes perfect sense and is a great test.

If RankBrain can improve results for queries that likely werent optimized for and will involve a mix of old and new entities and services a grouping of users who were likely getting lackluster results to begin with then it should be deployed globally.

ADVERTISEMENT

CONTINUE READING BELOW

Andit was in 2016.

Lets take a look at the two results I referenced above (and worth noting, I was writing the piece and the example and then thought to get the screen capture this is simply how it works and try it yourself it works in almost all cases where different wording implies the same thing):

Some very subtle differences in rankings with the #1 and 2 sites switching places but at its core its the same result.

Now lets look at my automotive example:

Machine learning helps Google to not just understand where there are similarities in queries, but we can also see it determining that if I need my car fixed I may need a mechanic (good call Google), whereas for replacing it I may be referring to parts or in need of governmental documentation to replace the entire thing.

ADVERTISEMENT

CONTINUE READING BELOW

We can also see here where machine learning hasnt quite figured it all out.

When I ask it how to replace my car, I likely mean the whole thing or Id have listed the part I wanted.

But itll learn its still in its infancy.

Also, Im Canadian, so the DMV doesnt really apply.

So here weve seen an example of machine learning at play in determining query meaning, SERP layout, and possible necessary courses of action to fulfill my intent.

Not all of that is RankBrain, but its all machine learning.

If you use Gmail, or pretty much any other email system, you also are seeing machine learning at work.

According to Google, they are now blocking 99.9% of all spam and phishing emails with a false-positive rate of only 0.05%.

Theyre doing this using the same core technique give the machine learning system some data and let it go.

If one was to manually program in all the permutations that would yield a 99.9% success rate in spam filtering and adjust on the fly for new techniques it would be an onerous task if at all possible.

ADVERTISEMENT

CONTINUE READING BELOW

When they did things this way they sat at a 97% success rate with 1% of false positive (meaning 1% of your real messages were sent to the spam folder unacceptable if it was important).

Enter machine learning set it up with all the spam messages you can positively confirm, let it build a model around what similarities they have, enter in some new messages and give it a reward for successfully selecting spam messages on its own and over time (and not a lot of it) it will learn far more signals and react far faster than a human ever could.

Set it to watch for user interactions with new email structures and when it learns that there is a new spam technique being used, add it to the mix and filter not just those emails but emails using similar techniques to the spam folder.

This article promised to be an explanation of machine learning, not just a list of examples.

ADVERTISEMENT

CONTINUE READING BELOW

The examples, however, were necessary to illustrate a fairly easy-to-explain model.

Lets not confuse this with easy to build, just simple in what we need to know.

A common machine learning model follows the following sequence:

This model is referred to as supervised learning and if my guess is right, its the model used in the majority of the Google algorithm implementations.

Another model of machine learning is the Unsupervised Model.

To draw from the example used in a great courseover on Coursera on machine learning, this is the model used to group similar stories in Google News and one can infer that its used in other places like the identification and grouping of images containing the same or similar people in Google Images.

In this model, the system is not told what its looking for but rather simply instructed to group entities (an image, article, etc.) into groups by similar traits (the entities they contain, keywords, relationships, authors, etc.)

ADVERTISEMENT

CONTINUE READING BELOW

Understanding what machine learning is will be crucial if you seek to understand why and how SERPs are laid out and why pages rank where they do.

Its one thing to understand an algorithmic factor which is an important thing to be sure but understanding the system in which those factors are weighted is of equal, if not greater, importance.

For example, if I was working for a company that sold cars I would pay specific attention to the lack of usable, relevant information in the SERP results to the query illustrated above.

The result is clearly not a success. Discover what content would be a success and generate it.

Pay attention to the types of content that Google feels may fulfill a users intent (post, image, news, video, shopping, featured snippet, etc.) and work to provide it.

I like to think of machine learning and its evolution equivalent to having a Google engineer sitting behind every searcher, adjusting what they see and how they see it before it is sent to their device.

ADVERTISEMENT

CONTINUE READING BELOW

But better that engineer is connected like the Borg to every other engineer learning from global rules.

But well get more into that in our next piece on user intent.

Image Credits

View post:
How Machine Learning in Search Works: Everything You Need to Know - Search Engine Journal

Every week, Analytics India Magazine reaches out to developers, practitioners and experts from the machine learning community to gain insights into their journey in data science, and the tools and skills essential for their day-to-day operations.

For this weeks column, Analytics India Magazine got in touch with Dipanjan Sarkar, a very well known face in the machine learning community. In this story, we take you through the journey of Dipanjan and how he became an ML expert.

Dipanjan currently works as a Data Science Lead at Applied Materials where he leads a team of data scientists to solve various problems in the manufacturing and semiconductor domain by leveraging machine learning, deep learning, computer vision and natural language processing. He provides the much needed technical expertise, AI strategy, solutioning, and architecture, and works with stakeholders globally.

He has a bachelors degree in computer science & engineering and a masters in data science from IIIT Bangalore. Currently, he is pursuing a PG Diploma in ML and AI from Columbia University and an executive education certification course in AI Strategy from Northwestern University Kellogg School of Management.

Apart from academia, Dipanjan is a big fan of MOOCs. He also beta-test new courses for Coursera before they are made public.

Dipanjan is also a Google Developer Expert in Machine Learning and has worked with several Fortune 500 companies. For an expert in ML, mathematics is a prerequisite, but we were surprised when we learnt that Dipanjan actually hated mathematics at school and this continued until ninth grade where he picked up statistics, linear algebra and calculus, the three pillars of machine learning.

I always loved the way you could program a computer to do specific tasks and make a machine actually learn with data!

Dipanjans renewed interest in mathematics was followed by his fascination for computer programming. With his growing fascination from mathematics to statistics and traditional computer programming, his career choice became almost obvious.

Reminiscing about his initial days, when the word data science wasnt worshipped yet, Dipanjan spoke about how the field was more conceptual and theoretical. Back then, there werent any active ecosystems of tools, languages and frameworks dedicated for data science. Hence, it took more time to learn theoretical concepts since it took more efforts to actually implement them or see them in practice.

With the advent of Python, R and a whole suite of tools and libraries, he believes that it has become easier to tame the learning curve of data science. However, he also warns that this can be a double-edged sword if one focuses on hands-on without deep-diving into the math and concepts behind algorithms and techniques to understand how it works or why it is used.

I have always been a strong advocate of self-learning, and I believe that is where you get maximum value

Due to the lack of mentors or proper guides, which are plenty nowadays on LinkedIn and other forums, Dipanjan had no other option than to self-learn with the help of the web and books.

For aspirants, he recommends the following books:

To dive deep into the concepts and to get hands-on, he recommends Deep Learning with Keras, Python Machine Learning and Hands-On Machine Learning as practical books with examples. Dipanjan has also written a handful of books on practical machine learning.

When it comes to practice and deploying ML models, Dipanjan extensively uses the CRISP-DM (cross industry process for data mining) framework, which he considers to be one of the best frameworks to tackle any data science problem.

Also, before diving into models or data, he insists on the importance of identifying and articulating the business problem in the right manner. For conceptualising an AI use-case, Dipanjan recommends something called AI Canvas, which he has learnt from the Kellogg School of Management:

Use the right tools for the job without waging wars of Python vs R or PyTorch vs TensorFlow

When asked about his favourite tools, Dipanjan explained the importance of not paying attention towards Python vs R or PyTorch vs TensorFlow and using the right tools that get the job done.

For instance, he and his team use the ecosystem of tools and libraries centered around Python very frequently. This includes the regular run-of-the-mill pandas, matplotlib, seaborn, plotly for data wrangling and exploratory data analysis. For statistical modelling he prefers libraries like scikit-learn, statsmodels and pyod.

Dipanjans toolkit looks as follows:

Along with picking the right tools, he recommends practitioners to always go with the simplest solution unless complexity is adding substantial value and last but not the least, he urges people not to ignore documentation.

To those looking to break into the world of data science, Dipanjan suggests one to follow a hybrid approach, i.e. learn concepts, code and apply them on real-world datasets.

First, learn all the math and concepts and then try to actually apply the methods you have learnt

In the long, tedious process of learning, Dipanjan warns that people might lose focus and get sidetracked into thinking why are they even learning a certain method. To remedy this, he insists on learning and applying if one aims of becoming a good data scientist without deviating from the goal.

Addressing the overwhelming hype around AI and ML, Dipanjan says that he is already witnessing the dust settling down and how companies are now actually starting to realise both the limitations and value of AI. Deep learning and deep transfer learning are actually starting to provide value for companies working on complex problems involving unstructured data like images, audio, video and text and things are only going to get bigger and better with advanced tools and hardware in future. However, he admits that there is definitely still a fair bit of hype out there.

Traditional machine learning models like linear and logistic regression will never go out of fashion

No matter how advanced the field gets, he believes that traditional machine learning models like linear and logistic regression will never go out of fashion since they are the bread and butter of various organisations and use-cases out there. And, models that are easy to explain, including linear models and decision trees will continue to be used extensively.

Going forward, he is optimistic about the use-cases and applications to optimise manufacturing, predicting demand and sales, inventory planning, logistics and routing, infrastructure management optimisation and enhancing customer support and experience, will continue to be the key drivers for almost all major organisations for the next decade.

When it comes to breakthroughs, Dipanjan expects something big to happen in newer domains like self-learning, continuous-learning, meta-learning and reinforcement learning.

Always remember to challenge others opinions with a healthy mindset because a good data scientist doesnt just follow instructions blindly.

Talking about his tireless efforts to guide youngsters, he recollects how not having a mentor had been a major hindrance and how he had to unlearn and relearn overtime to correct his misconceptions. To help aspirants avoid the same mistakes, he mentors them whenever possible.

On a concluding note, Dipanjan said that he is mightily impressed by the relentless efforts of the data science community to share ideas through blogs, vlogs and online forums. Confessing his love for Analytics India Magazine, Dipanjan spoke about how AIM has been fostering a rich analytics ecosystem in India by reaching out to the global community.

Dipanjan will be speaking at Analytics India Magazines inaugural virtual conference, Plugin on 28th of May 2020. For more information, check our portal here.

comments

Original post:
The ML Expert Who Hated Mathematics: Interview With Dipanjan Sarkar - Analytics India Magazine

Impetus TechnologiesInc., a leading software products and services company, announced the launch of its newcloud-based version of StreamAnalytixonAWS Marketplace. StreamAnalytix Cloud will also be available on other leading cloud marketplaces like Azure and Google Cloud very soon.

Customer-driven Open Source Technology Platform is Future

Leveraging an interactive data-first approach, the tool provides an intuitive drag-and-drop interface to build ETL flows on the cloud, effortlessly. Users can ingest data from multiple on-premise and cloud sources, enrich this data, and swiftly build applications for a wide range of analytics use cases.

StreamAnalytix Cloud offers a host of power-packed features, including:

We are focused on helping enterprises harness the limitless power of the cloud to build, test, and run ETL and machine learning applications faster across industries and use cases, saidPunit Shah, Director for StreamAnalytix at Impetus Technologies. As a next-generation ETL tool in the cloud,StreamAnalytix Cloud accelerates Spark application development, and empowers users with unmatched scalability and extensibility to meet their strategic business needs.

4 Cyber Security Predictions to Watch Out for in 2020

Originally posted here:
Impetus StreamAnalytix Launches a Cloud-based Version for Self-service ETL and Machine Learning - EnterpriseTalk

The next progression in organizations using threat intelligence is adding AI threat intelligence capabilities, in the form of machine learning technologies, to improve attack detection. Machine learning is a form of AI that enables computers to analyze data and learn its significance. The rationale for using machine learning with threat intelligence is to enable computers to more rapidly detect attacks than humans can and stop those attacks before more damage occurs. In addition, because the volume of threat intelligence is often so large, traditional detection technologies inevitably generate too many false positives. Machine learning can analyze the threat intelligence and condense it into a smaller set of things to look for, thereby reducing the number of false positives.

This sounds fantastic, but there's a catch -- actually, a few catches. Expecting AI to magically improve security is unrealistic, and deploying machine learning without preparation and ongoing support may make things worse.

Here are three steps enterprises should take to use AI threat intelligence tools with machine learning capabilities to improve attack detection.

AI threat intelligence products that use machine learning work by taking inputs, analyzing them and producing outputs. For attack detection, machine learning's inputs include threat intelligence, and its outputs are either alerts indicating attacks or automated actions stopping attacks. If the threat intelligence has errors, it will give "bad" information to the attack detection tools, so the tools' machine learning algorithms may produce "bad" outputs.

Many organizations subscribe to multiple sources of threat intelligence. These include feeds, which contain machine-readable signs of attacks, like the IP addresses of computers issuing attacks and the file names used by malware. Other sources of threat intelligence are services, which generally provide human-readable prose describing the newest threats. Machine learning can use feeds but not services.

Organizations should use the highest quality threat intelligence feeds for machine learning. Characteristics to consider include the following:

It's hard to directly evaluate the quality of threat intelligence, but you can indirectly evaluate it based on the number of false positives that occur from using it. High-quality threat intelligence should lead to minimal false positives when it's used directly by detection tools -- without machine learning.

False positives are a real concern if you're using threat intelligence with machine learning to do things like automatically block attacks. Mistakes will disrupt benign activity and could negatively affect operations.

Ultimately, threat intelligence is just one part of assessing risk. Another part is understanding context -- like the role, importance and operational characteristics of each computer. Providing contextual information to machine learning can help it get more value from threat intelligence. Suppose threat intelligence indicates a particular external IP address is malicious. Detecting outgoing network traffic from an internal database server to that address might merit a different action than outgoing network traffic to the same address from a server that sends files to subscribers every day.

The toughest part of using machine learning is providing the actual learning. Machine learning needs to be told what's good and what's bad, as well as when it makes mistakes so it can learn from them. This requires frequent attention from skilled humans. A common way of teaching machine learning-enabled technologies is to put them into a monitor-only mode where they identify what's malicious but don't block anything. Humans review the machine learning tool's alerts and validate them, letting it know which were erroneous. Without feedback from humans, machine learning can't improve on its mistakes.

Conventional wisdom is to avoid relying on AI threat intelligence that uses machine learning to detect attacks because of concern over false positives. That makes sense in some environments, but not in others. Older detection techniques are more likely to miss the latest attacks, which may not follow the patterns those techniques typically look for. Machine learning can help security teams find the latest attacks, but with potentially higher false positive rates. If missing attacks is a greater concern than the resources needed to investigate additional false positives, then more reliance on automation utilizing machine learning may make sense to protect those assets.

Many organizations will find it best to use threat intelligence without machine learning for some purposes, and to get machine learning-generated insights for other purposes. For example, threat hunters might use machine learning to get suggestions of things to investigate that would have been impossible for them to find in large threat intelligence data sets. Also, don't forget about threat intelligence services -- their reports can provide invaluable insights for threat hunters on the newest threats. These insights often include things that can't easily be automated into something machine learning can process.

Originally posted here:
AI threat intelligence is the future, and the future is now - TechTarget

Complex machines that steer autonomous vehicles, set the temperature in our homes and buy and sell stocks with little human control are built to learn from their environments and act on what they see or hear. They can be tricked into grave errors by relatively simple attacks or innocent misunderstandings, but they may be able to help themselves by mixing their senses.

In 2018, a group of security researchers managed to befuddle object-detecting software with tactics that appear so innocuous its hard to think of them as attacks. By adding a few carefully designed stickers to stop signs, the researchers fooled the sort of object-recognizing computer that helps guide driverless cars. The computers saw an umbrella, bottle or banana but no stop sign.

Two multi-colored stickers attached to a stop sign were enough to disguise it to the eyes of an image-recognition algorithm as a bottle, banana and umbrella. UW-Madison

They did this attack physically added some clever graffiti to a stop sign, so it looks like some person just wrote on it or something and then the object detectors would start seeing it is a speed limit sign, says Somesh Jha, a University of WisconsinMadison computer sciences professor and computer security expert. You can imagine that if this kind of thing happened in the wild, to an auto-driving vehicle, that could be really catastrophic.

The Defense Advanced Research Projects Agency has awarded a team of researchers led by Jha a $2.7 million grant to design algorithms that can protect themselves against potentially dangerous deception. Joining Jha as co-investigators are UWMadison Electrical and Computer Engineering Professor Kassem Fawaz, University of Toronto Computer Sciences Professor Nicolas Papernot, and Atul Prakash, a University of Michigan professor of Electrical Engineering and Computer Science and an author of the 2018 study.

Kassem Fawaz

One of Prakashs stop signs, now an exhibit at the Science Museum of London, is adorned with just two narrow bands of disorganized-looking blobs of color. Subtle changes can make a big difference to object- or audio-recognition algorithms that fly drones or make smart speakers work, because they are looking for subtle cues in the first place, Jha says.

The systems are often self-taught through a process called machine learning. Instead of being programmed into rigid recognition of a stop sign as a red octagon with specific, blocky white lettering, machine learning algorithms build their own rules by picking distinctive similarities from images that the system may know only to contain or not contain stop signs.

The more examples it learns from, the more angles and conditions it is exposed to, the more flexible it can be in making identifications, Jha says. The better it should be at operating in the real world.

But a clever person with a good idea of how the algorithm digests its inputs might be able to exploit those rules to confuse the system.

DARPA likes to stay a couple steps ahead, says Jha. These sorts of attacks are largely theoretical now, based on security research, and wed like them to stay that way.

A military adversary, however or some other organization that sees advantage in it could devise these attacks to waylay sensor-dependent drones or even trick largely automated commodity-trading computers run into bad buying and selling patterns.

Somesh Jha

What you can do to defend against this is something more fundamental during the training of the machine learning algorithms to make them more robust against lots of different types of attacks, says Jha.

One approach is to make the algorithms multi-modal. Instead of a self-driving car relying solely on object-recognition to identify a stop sign, it can use other sensors to cross-check results. Self-driving cars or automated drones have cameras, but often also GPS devices for location and laser-scanning LIDAR to map changing terrain.

So, while the camera may be saying, Hey this is a 45-mile-per-hour speed limit sign, the LIDAR says, But wait, its an octagon. Thats not the shape of a speed limit sign, Jha says. The GPS might say, But were at the intersection of two major roads here, that would be a better place for a stop sign than a speed limit sign.

The trick is not to over-train, constraining the algorithm too much.

The important consideration is how you balance accuracy against robustness against attacks, says Jha. I can have a very robust algorithm that says every object is a cat. It would be hard to attack. But it would also be hard to find a use for that.

Share via Facebook

Share via Twitter

Share via Linked In

Share via Email

See the rest here:
Teaching machine learning to check senses may avoid sophisticated attacks - University of Wisconsin-Madison