Julien Rebetez, is the Lead Software & Machine Learning Engineer at Picterra. Picterra provides a geospatial cloud-based-platform specially designed for training deep learning based detectors, quickly and securely.
Without a single line of code and with only few human-made annotations, Picterras users build and deploy unique actionable and ready to use deep learning models.
Itautomates the analysis of satellite and aerial imagery, enabling users to identify objects and patterns.
What is it that attracted you to machine learning and AI?
I started programming because I wanted to make video games and got interested in computer graphics at first. This led me to computer vision, which is kind of the reverse process where instead of having the computer create a fake environment, you have it perceive the real environment. During my studies, I took some Machine Learning courses and I got interested in the computer vision angle of it. I think whats interesting about ML is that its at the intersection between software engineering, algorithms and math and it still feels kind of magical when it works.
Youve been working on using machine learning to analyze satellite image for many years now. What was your first project?
My first exposure to satellite imagery was the Terra-i project (to detect deforestation) and I worked on it during my studies. I was amazed at the amount of freely available satellite data that is produced by the various space agencies (NASA, ESA, etc). You can get regular images of the planet for free every day or so and this is a great resource for many scientific applications.
Could you share more details regarding the Terra-i project?
The Terra-i project (http://terra-i.org/terra-i.html) was started by Professor Andrez Perez-Uribe, from HEIG-VD (Switzerland) and is now led by Louis Reymondin, from CIAT (Colombia). The idea of the project is to detect deforestation using freely available satellite images. At the time, we worked with MODIS imagery (250m pixel resolution) because it provided a uniform and predictable coverage (both spatially and temporally). We would get a measurement for each pixel every few days and from this time series of measurements, you can try to detect anomalies or novelties as we call them in ML sometimes.
This project was very interesting because the amount of data was a challenge at the time and there was also some software engineering involved to make it work on multiple computers and so on. From the ML side, it used Bayesian Neural Network (not very deep at the time ) to predict what the time series of a pixel should look like. If the measurement didnt match the prediction, then we would have an anomaly.
As part of this project, I also worked on cloud removal. We took a traditional signal processing approach there, where you have a time series of measurements and some of them will be completely off because of a cloud. We used a fourier-based approach (HANTS) to clean the time series before detecting novelties in it. One of the difficulties is that if we would clean it too strongly, wed also remove novelties, so there were quite some experiments to do to find the right parameters.
You also designed and implemented a deep learning system for automatic crop type classification from aerial (drone) imagery of farm fields. What were the main challenges at the time?
This was my first real exposure to Deep Learning. At the time, I think the main challenge were more on getting the framework to run and properly use a GPU than on the ML itself. We used Theano, which was one of the ancestors of Tensorflow.
The goal of the project was to classify the type of crop in a field, from drone imagery. We tried an approach where the Deep Learning Model was using color histograms as inputs as opposed to just the raw image. To make this work reasonably quickly, I remember having to implement a custom Theano layer, all the way to some CUDA code. That was a great learning experience at the time and a good way to dig a bit into the technical details of Deep Learning.
Youre officially the Lead Software and Machine Learning Engineer at Picterra. How would you best describe your day to day activities?
It really varies, but a lot of it is about keeping an eye on the overall architecture of the system and the product in general and communicating with the various stakeholders. Although ML is at the core of our business, you quickly realize that most of the time is not spent on ML itself, but all the things around it: data management, infrastructure, UI/UX, prototyping, understanding users, etc This is quite a change from Academia or previous experience in bigger companies where you are much more focused on a specific problem.
Whats interesting about Picterra is that we not only run Deep Learning Models for users, but we actually allow them to train their own. That is different from a lot of the typical ML workflows where you have the ML team train a model and then publish it to production. What this means is that we cannot manually play with the training parameters as you often do. We have to find some training method that will work for all of our users. This led us to create what we call our experiment framework, which is a big repository of datasets that simulates the training data our users would build on the platform. We can then easily test changes to our training methodology against these datasets and evaluate if they help or not. So instead of evaluating a single model, we are more evaluating an architecture + training methodology.
The other challenge is that our users are not ML practitioners, so they dont necessarily know what a training set is, what a label is and so on. Building a UI to allow non-ML practitioners to build datasets and train ML models is a constant challenge and there is a lot of back-and-forth between the UX and ML teams to make sure we guide users in the right direction.
Some of your responsibilities include prototyping new ideas and technologies. What are some of the more interesting projects that you have worked on?
I think the most interesting one at Picterra was the Custom Detector prototype. 1.5 years ago, we had built-in detectors on the platform: those were detectors that we trained ourselves and made accessible to users. For example, we had a building detector, a car detector, etc
This is actually the typical ML workflow: you have some ML engineer develop a model for a specific case and then you serve it to your clients.
But we wanted to do something differently and push the boundaries a bit. So we said: What if we allow users to train their own models directly on the platform ? There were a few challenges to make this work: first, we didnt want this to take multiple hours. If you want to keep this feeling of interactivity, training should take a few minutes at most. Second, we didnt want to require thousands of annotations, which is typically what you need for large Deep Learning models.
So we started with a super simple model, did a bunch of tests in jupyter and then tried to integrate it in our platform and test the whole workflow, with a basic UI and so on. At first, it wasnt working very well in most cases, but there were a few cases where it would work. This gave us hope and we started iterating on the training methodology and the model. After some months, we were able to reach a point where it worked well, and we now have our users using this all the time.
What was interesting about this is the double challenge of keeping the training fast (currently a few minutes) and therefore the model not too complex, but at the same time making it complex enough that it works and solves users problems. On top of that, it works with few (<100) labels for a lot of cases.
We also applied many of Googles Rules of Machine Learning, in particular the ones about implementing the whole pipeline and metrics before starting to optimize the model. It puts you into system thinking mode where you figure out that not all your problems should be handled by the core ML, but some of them can be pushed to the UI, some of them pre/post-processed, etc
What are some of the machine learning technologies that are used at Picterra?
In production, we are currently using Pytorch to train & run our models. We are also using Tensorflow from time to time, for some specific models developed for clients. Other than that, its a pretty standard scientific Python stack (numpy, scipy) with some geospatial libraries (gdal) thrown in.
Can you discuss how Picterra works in the backend once someone uploads images and wishes to train the neural network to properly annotate objects?
Sure, so first when you upload an image, we process it and store it in a Cloud-Optimized-Geotiff (COG) format on our blobstore (Google Cloud Storage), which allows us to quickly access blocks of the image without having to download the whole image later on. This is a key point because geospatial imagery can be huge: we have users routinely working with 5000050000 images.
So then, to train your model, you will have to create your training dataset through our web UI. You will do that by defining 3 types of areas:
Once you have created this dataset, you can simply click Train and well train a detector for you. What happens next is that we enqueue a training job, have one of our GPU worker pick it up (new GPU workers are started automatically if there are many concurrent jobs), train your model, save its weights to the blobstore and finally predict in the testing area to display on the UI. From there, you can iterate over your model. Typically, youll spot some mistakes in testing areas and add training areas to help the model improve.
Once you are happy with the score of your model, you can run it at scale. From the users point of view, this is really simple: just click on Detect next to the image you want to run it on. But its a bit more involved under the hood if the image is large. To speed things up, handle failures and avoid having detections taking multiple hours, we break down large detections in grid cells and run an independent detection job for each cell. This allows us to run very large-scale detections. For example, we had a customer run detection over the whole country of Denmark on 25cm imagery, which is in the range of TB of data for a single project. Weve covered a similar project in this medium post.
Is there anything else that you would like to share about Picterra?
I think whats great about Picterra is that it is a unique product, at the intersection between ML and Geospatial. What differentiates us from other companies that process geospatial data is that we equip our users with a self-serve platform. They can easily find locations, analyze patterns, and detect and count objects on Earth observation imagery. It would be impossible without machine learning, but our users dont even need basic coding skills the platform does the work based on a few human-made annotations. For those who want to go deeper and learn the core concepts of machine learning in the geospatial domain, we have launched a comprehensive online course.
What is also worth mentioning is that possible applications of Picterra are endless detectors built on the platform have been used in city management, precision agriculture, forestry management, humanitarian and disaster risk management, farming, etc., just to name the most common applications. We are basically surprised every day by what our users are trying to do with our platform. You can give it a try and let us know how it worked on social media.
Thank you for the great interview and for sharing with us how powerful Picterra is, readers who wish to learn more should visit the Picterra website.
Read this article:
- Integrating Artificial Intelligence in Treatment Planning - Imaging Technology News - July 8th, 2020
- Viz.ai Named to Forbes AI 50 List of Most Promising Artificial Intelligence Companies - Yahoo Finance - July 8th, 2020
- COVID-19 Impact and Recovery Analysis | Artificial Intelligence (AI) Market In BFSI Sector 2019-2023 | Focus On Autonomous Banking to Boost Growth |... - July 8th, 2020
- COVID-19 and Artificial Intelligence: How the pandemic has re-ignited a focus on the software - Savannah Morning News - July 8th, 2020
- Artificial Intelligence: worth the hype? - BusinessCloud - July 8th, 2020
- Artificial Intelligence Powered COVID-19 Diagnosis Tool to Seek Lung Scan Images from India - The Weather Channel - July 8th, 2020
- How Artificial Intelligence Could Lead to Better Investment Decisions - Barron's - July 8th, 2020
- Artificial intelligence should be used to augment human creativity - not replace it - Bizcommunity.com - July 8th, 2020
- LI startup predicts where COVID-19 will spike - Newsday - July 8th, 2020
- Forums held ahead of World Artificial Intelligence Conference - Chinadaily USA - July 8th, 2020
- VeChain Is Attending the World Artificial Intelligence Conference 2020 Hosted - AiThority - July 8th, 2020
- ContractPodAi and Bowmans Partner to Bring Artificial Intelligence-Powered Contract Management to the African Market - Business Wire - July 8th, 2020
- The Amalgamation of Human Brain and Artificial Intelligence - Analytics Insight - July 8th, 2020
- The main beneficiaries of artificial intelligence success are IT departments themselves - ZDNet - July 8th, 2020
- Artificial intelligence isn't all about the Terminator, tech sceptics are warned - Mirror Online - July 8th, 2020
- How Machine Learning Will Impact the Future of Software Development and Testing - ReadWrite - July 8th, 2020
- Artificial Intelligence (AI) Definition - Investopedia - May 31st, 2020
- The Temptations Of Artificial Intelligence Technology And The Price Of Admission - Forbes - May 31st, 2020
- Artificial intelligence offers a chance to optimize COVID-19 treatment in international partnership - Vanderbilt University News - May 31st, 2020
- Reality Check: The Benefits of Artificial Intelligence - AiThority - May 31st, 2020
- Thanks To Renewables And Machine Learning, Google Now Forecasts The Wind - Forbes - May 31st, 2020
- Artificial intelligence is hopelessly biased - and that's how it will stay - TechRadar India - May 31st, 2020
- Top Artificial Intelligence Trends that will Change the Decade - Analytics Insight - May 31st, 2020
- Art and artifice - The Indian Express - May 31st, 2020
- The Role of Artificial Intelligence in Ethical Hacking | EC-Council Official Blog - EC-Council Blog - May 31st, 2020
- Microsoft is cutting dozens of MSN news production workers and replacing them with artificial intelligence - Seattle Times - May 31st, 2020
- Artificial intelligence software improves accuracy, doubles speed in evaluating CT scans of advanced cancer - UAB News - May 31st, 2020
- Coronavirus tests the value of artificial intelligence in medicine - FierceHealthcare - May 31st, 2020
- The Use of Artificial Intelligence by Investment Advisers: Considerations Based on an Advisers Fiduciary Duties - JD Supra - May 31st, 2020
- Lecturer in Artificial Intelligence for Digital Infrastructures job with UNIVERSITY OF BRISTOL | 208485 - Times Higher Education (THE) - May 31st, 2020
- Artificial Intelligence (AI) Market to Reach USD 202.57 Billion by 2026; Rising Demand for Cloud-based Applications to Aid Growth: Fortune Business... - May 31st, 2020
- Walmart Employees Are Out to Show Its Anti-Shoplifting AI Doesn't Work - WIRED - May 31st, 2020
- How artificial intelligence is keeping time-critical shipments on track during pandemic - FreightWaves - May 31st, 2020
- Coronavirus Update: Recent FTC Guidance on the Use of Artificial Intelligence and Algorithms in the Age of COVID-19 - Government Contracts Legal Forum - May 31st, 2020
- COVID-19 Impact: A Mix of Challenges and Opportunities | Artificial Intelligence-as-a-Service (AIaaS) Market 2020-2024 | Growing Adoption of Cloud... - May 31st, 2020
- IIT-Ropar and TSW Launch a PG Programme in Artificial Intelligence - THE WEEK - May 31st, 2020
- MS in Artificial Intelligence | Artificial Intelligence - May 19th, 2020
- What is Artificial Intelligence? | Azure Blog and Updates ... - May 19th, 2020
- What Are the Advantages of Artificial Intelligence ... - May 19th, 2020
- AI Tutorial | Artificial Intelligence Tutorial - Javatpoint - May 19th, 2020
- It's Called Artificial Intelligencebut What Is Intelligence? - WIRED - May 19th, 2020
- Powering the Artificial Intelligence Revolution - HPCwire - May 19th, 2020
- Artificial intelligence is struggling to cope with how the world has changed - ZDNet - May 19th, 2020
- An AI future set to take over post-Covid world - The Indian Express - May 19th, 2020
- A New Way To Think About Artificial Intelligence With This ETF - MarketWatch - May 19th, 2020
- Artificial intelligence-based imaging reconstruction may lead to incorrect diagnoses, experts caution - Radiology Business - May 19th, 2020
- Artificial Intelligence in the Covid Frontline - Morningstar - May 19th, 2020
- Patent Analytics Market to Reach USD 1,668.4 Million by 2027; Integration of Machine Learning and Artificial Intelligence to Spur Business... - May 19th, 2020
- Artificial Intelligence in Cancer: How Is It Used in Practice? - Cancer Therapy Advisor - May 19th, 2020
- Five Important Subsets of Artificial Intelligence - Analytics Insight - May 19th, 2020
- Ethical artificial intelligence: Could Switzerland take the lead? - swissinfo.ch - May 19th, 2020
- The Future of Artificial Intelligence: Edge Intelligence - Analytics Insight - May 19th, 2020
- 5 Ways Artificial Intelligence Is Changing Architecture - May 15th, 2020
- Keysight Validates Artificial Intelligence (AI) and ... - May 15th, 2020
- What Is Artificial Intelligence? Examples and News in 2019 ... - May 15th, 2020
- Artificial Intelligence (AI) Is Nothing Without Humans - E3zine.com - May 15th, 2020
- Artificial intelligence is helping seniors who are isolated during the coronavirus pandemic - WXYZ - May 15th, 2020
- ValleyML is launching a Machine Learning and Deep Learning Boot Camp from July 14th to Sept 10th and AI Expo Series from Sept 21st to Nov 19th 2020.... - May 15th, 2020
- TSA Issues Road Map to Tackle Insider Threat With Artificial Intelligence - Nextgov - May 15th, 2020
- The Expanding Role Of Artificial Intelligence In Tax - Forbes - May 15th, 2020
- Artificial Intelligence: Technology Of The Future - Forbes India - May 15th, 2020
- SparkCognition and Milize to Offer Automated Machine Learning Solutions for Financial Institutions to the APAC Region - PRNewswire - May 15th, 2020
- Alpha Modus Allowed 2nd Patent Leveraging Artificial Intelligence for Monitoring and Analyzing Consumer Behavior - Yahoo Finance - May 15th, 2020
- Tesla : Artificial Intelligence - the .AI domain is on the rise - marketscreener.com - May 15th, 2020
- Artificial Intelligence Can Only Help Architecture if We Ask the Right Questions - ArchDaily - May 15th, 2020
- Fighting COVID-19: Artificial Intelligence community to help citizens and the healthcare system - Canada NewsWire - May 15th, 2020
- Understanding the Four Types of Artificial Intelligence - May 9th, 2020
- Artificial intelligence Facts for Kids - May 9th, 2020
- 4 Main Types of Artificial Intelligence - G2 - May 9th, 2020
- Artificial intelligence project lets Holocaust survivors ... - May 9th, 2020
- Artificial intelligence - Simple English Wikipedia, the ... - May 9th, 2020
- Artificial Intelligence Market by Size, Share, Analysis ... - May 9th, 2020
- A.I. Artificial Intelligence movie review (2001) | Roger Ebert - May 9th, 2020
- Artificial Intelligence is Evolving to Process the World Like Humans - Interesting Engineering - May 9th, 2020
- Drug research turns to artificial intelligence in COVID-19 fight - Business in Vancouver - May 9th, 2020
- Women wanted: Why now could be a good time for women to pursue a career in AI - CNBC - May 9th, 2020
- The Impending Artificial Intelligence Revolution in Healthcare - Op-Ed - HIT Consultant - May 9th, 2020
- COVID-19 identification in X-ray images by Artificial intelligence - News Anyway - May 9th, 2020
- Is artificial intelligence the answer to the care sector amid COVID-19? - Descrier - May 9th, 2020
- Artificial intelligence - Wikipedia - April 26th, 2020