Author : Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Brgier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, Jrme Revaud
Abstract : Despite impressive performance for high-level downstream tasks, self-supervised pre-training methods have not yet fully delivered on dense geometric vision tasks such as stereo matching or optical flow. The application of selfsupervised concepts, such as instance discrimination or masked image modeling, to geometric tasks is an active area of research. In this work, we build on the recent crossview completion framework, a variation of masked image modeling that leverages a second view from the same scene which makes it well suited for binocular downstream tasks. The applicability of this concept has so far been limited in at least two ways: (a) by the difficulty of collecting realworld image pairs in practice only synthetic data have been used and (b) by the lack of generalization of vanilla transformers to dense downstream tasks for which relative position is more meaningful than absolute position. We explore three avenues of improvement: first, we introduce a method to collect suitable real-world image pairs at large scale. Second, we experiment with relative positional embeddings and show that they enable vision transformers to perform substantially better. Third, we scale up vision transformer based cross-completion architectures, which is made possible by the use of large amounts of data. With these improvements, we show for the first time that stateof-the-art results on stereo matching and optical flow can be reached without using any classical task-specific techniques like correlation volume, iterative estimation, image warping or multi-scale reasoning, thus paving the way towards universal vision models.
2. Self-Supervised Intensity-Event Stereo Matching(arXiv)
Author : Jinjin Gu, Jinan Zhou, Ringo Sai Wo Chu, Yan Chen, Jiawei Zhang, Xuanye Cheng, Song Zhang, Jimmy S. Ren
Abstract : Event cameras are novel bio-inspired vision sensors that output pixel-level intensity changes in microsecond accuracy with a high dynamic range and low power consumption. Despite these advantages, event cameras cannot be directly applied to computational imaging tasks due to the inability to obtain high-quality intensity and events simultaneously. This paper aims to connect a standalone event camera and a modern intensity camera so that the applications can take advantage of both two sensors. We establish this connection through a multi-modal stereo matching task. We first convert events to a reconstructed image and extend the existing stereo networks to this multi-modality condition. We propose a self-supervised method to train the multi-modal stereo network without using ground truth disparity data. The structure loss calculated on image gradients is used to enable self-supervised learning on such multi-modal data. Exploiting the internal stereo constraint between views with different modalities, we introduce general stereo loss functions, including disparity cross-consistency loss and internal disparity loss, leading to improved performance and robustness compared to existing approaches. The experiments demonstrate the effectiveness of the proposed method, especially the proposed general stereo loss functions, on both synthetic and real datasets. At last, we shed light on employing the aligned events and intensity images in downstream tasks, e.g., video interpolation application.
Read the original:
Use cases of Stereo Matching part7(Machine Learning + AI) - Medium
- Predictive Analytics And Machine Learning Market: A ... - Fagen wasanni - August 4th, 2023 [August 4th, 2023]
- Photonic Neural Networks: Revolutionizing Machine Learning and AI - Fagen wasanni - August 4th, 2023 [August 4th, 2023]
- Growing Concerns Over Bias in Powerful AI and Machine Learning ... - Fagen wasanni - August 4th, 2023 [August 4th, 2023]
- Machine learning prediction and classification of behavioral ... - Nature.com - August 4th, 2023 [August 4th, 2023]
- Predicting BRAFV600E mutations in papillary thyroid carcinoma ... - Nature.com - August 4th, 2023 [August 4th, 2023]
- Johns Hopkins makes major investment in the power, promise of ... - The Hub at Johns Hopkins - August 4th, 2023 [August 4th, 2023]
- Postdoctoral Fellowship: Pathogenesis of High Consequence ... - Global Biodefense - August 4th, 2023 [August 4th, 2023]
- Apple's Commitment to Generative AI and Machine Learning - Fagen wasanni - August 4th, 2023 [August 4th, 2023]
- Richmond could become AI and machine learning tech hub - The Daily Progress - August 4th, 2023 [August 4th, 2023]
- Platform Reduces Barriers Biologists Face In Accessing Machine ... - Bio-IT World - August 4th, 2023 [August 4th, 2023]
- A comparative study of predicting the availability of power line ... - Nature.com - August 4th, 2023 [August 4th, 2023]
- Preventing Bias In Machine Learning - Texas A&M Today - Texas A&M University Today - August 4th, 2023 [August 4th, 2023]
- 3 Cheap Machine Learning Stocks That Smart Investors Will Snap ... - InvestorPlace - August 4th, 2023 [August 4th, 2023]
- Research Analyst/ Associate/ Fellow in Machine Learning and ... - Times Higher Education - August 6th, 2023 [August 6th, 2023]
- AI and Machine Learning: The New Frontier in Global Anti-Money ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- Harnessing the Power of AI and Machine Learning: Growth ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- Harnessing the Power of AI and Machine Learning for Enhanced ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- Use cases of Stereo Matching part8(Machine Learning + AI) - Medium - August 6th, 2023 [August 6th, 2023]
- Use cases of Stereo Matching part9(Machine Learning + AI) - Medium - August 6th, 2023 [August 6th, 2023]
- How machine learning can expand the Landscape of Edge AI. | TDK - TDK Corporation - August 6th, 2023 [August 6th, 2023]
- Machine Learning-Trained Autonomy Tested By XQ-58 For Skyborg - Aviation Week - August 6th, 2023 [August 6th, 2023]
- Artificial Intelligence and Machine Learning in Packaging Robotics ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- 86-year old Hammett equation gets a machine learning update - Chemistry World - August 6th, 2023 [August 6th, 2023]
- Q & A: How A.I. and machine learning are transforming the lending ... - Digital Journal - August 6th, 2023 [August 6th, 2023]
- The Rise of AI and Machine Learning in Global E-Commerce ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- Machine learning-based technique for gain and resonance ... - Nature.com - August 6th, 2023 [August 6th, 2023]
- Machine learning for the development of diagnostic models of ... - Nature.com - August 6th, 2023 [August 6th, 2023]
- AI and the Heart: How Machine Learning is Changing the Face of ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- The Hidden Impact of AI in Photography and How Machine Learning ... - Cryptopolitan - August 6th, 2023 [August 6th, 2023]
- Machine learning identifies physical signs of stroke - Open Access Government - August 6th, 2023 [August 6th, 2023]
- Machine-learning for the prediction of one-year seizure recurrence ... - Nature.com - August 6th, 2023 [August 6th, 2023]
- Automated Machine Learning: Revolutionizing Predictive Analytics ... - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- Tim Cook says AI, machine learning are part of virtually every product Apple is building - CryptoSlate - August 6th, 2023 [August 6th, 2023]
- AI GNNs: Transforming the Landscape of Machine Learning - Fagen wasanni - August 6th, 2023 [August 6th, 2023]
- 3 Cheap Machine Learning Stocks That Smart Investors Will Snap Up Now - InvestorPlace - August 6th, 2023 [August 6th, 2023]