Cool tools

Collection of useful tools for students, young researchers and enthousiasts in cognitive sciences and data science. There are already a lot of ressources everywhere arround the internet but here will be my personal collection of tools.

Machine Learning

BOOKS

ARTICLES

• Historical Papers:
• Computing Machinery and Intelligence, Alan Turing, 1950 (section 7!)
• Understanding natural language, Terry Winograd, 1972 (SHRUDLU / Put that there)
• Intelligence without representation, Rodney Brooks, 1991
• Classical Learning:
• Supervised
• Regression
• Classification
• Self-Supervised (Unsupervised)
• SOM
• Self-Organizing Maps, Teuvo Kohonen, 1982
• Blog post: Kohonen's Self Organizing Feature Maps, ai-junkie
• Blog post: Creating hexagonal heatmaps with d3.js, Nadieh Bremer, 2013
• Boltzman Machines
• Greedy Layer-Wise Training of Deep Networks, Yoshua Bengio, 2006
• A fast learning algorithm for deep belief nets, Geoffrey E. Hinton, 2006
• Deep Boltzmann Machine, Ruslan Salakhutdinov, 2009
• AutoEncoders
• Reducing the Dimensionality of Data with Neural Networks, Ruslan Salakhutdinov, Geoffrey E. Hinton, 2006
• Extracting and Composing Robust Features with Denoising Autoencoders, P Vincent, Larochelle, Bengio, 2008
• Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion, P Vincent, Larochelle, Bengio, Lajoie, 2010
• Contractive Auto-Encoders:Explicit Invariance During Feature Extraction, S Rifai, Glorot, Bengio, 2011
• k-Sparse Autoencoders, Alireza Makhzanio, 2013
• Blog Post: Deep Learning Tutorial - Sparse Autoencoder, Chris McCormick, 2014
• Blog Post: Building Autoencoders in Keras, Francois Chollet, 2016
• Ensemble:
• Bagging
• Bagging predictors, L Breiman, 2016
• Boosting
• A decision-theoretic generalization of on-line learning and an application to boosting, Y Freund, R Schapire, 1997 (AdaBoost, Freund and Schapire won the 2003 Gödel Prize for their work)
• The return of AdaBoost.MH: multi-class Hamming trees, Balázs Kégl, 2013
• Stacking
• Stacked Generalization, Wolpert, 1992
• Blog Post: Ensemble methods: bagging, boosting and stacking, Joseph Rocca, 2019
• Artificial Neural Networks and Deep Learning:
• ANN
• Efficient BackProp, LeCun, 1998
• Deep Sparse Rectifier Neural Networks, Glorot, Bordes, Bengio, 2010
• Dropout: A Simple Way to Prevent Neural Networks from, N Srivastava, 2014
• Neural Networks and Deep Learning, Nielsen, 2015
• Uncertainty in Deep Learning, Yarin Gal, 2016 (shows that a lot of ML techniques can be re interpreted in Bayesian terms like the dropout for instance)
• CNN
• Gradient-based learning applied to document recognition, LeCun, 1994
• Evaluation of pooling operations in convolutional architectures for object recognition, Dominik Scherer, 2010
• ImageNet Classification with Deep Convolutional Neural Networks (AlexNet), Krizhevsky, Sutskever, Hinton, 2012
• Very Deep Convolutional Networks for Large-Scale Image Recognition, Simonyan, 2014 (VggNet)
• Going Deeper with Convolutions (GoogleLeNet), Christian Szegedy, 2014
• Deep Residual Learning for Image Recognition (ResNet), Kaiming He, 2015
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, Kaiming He, 2015
• Understanding Convolutional Neural Networks with A Mathematical Model, C.-C. Jay Kuo, 2016
• Blog post: A Friendly Introduction to Cross-Entropy Loss, Rob DiPietro, 2016
• Introduction to Convolutional Neural Networks, Jianxin Wu, 2017
• Squeeze-and-Excitation Networks (SENet), Jie Hu, 2017
• Blog post: The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3), Adit Deshpande, 2019
• Objects Detection
• You Only Look Once: Unified, Real-Time Object Detection (YOLO), Joseph Redmon, 2015
• Semantic Segmantation
• U-Net: Convolutional Networks for Biomedical Image Segmentation (U-Net), Olaf Ronneberger, 2015
• Super Resolution
• Enhanced Deep Residual Networks for Single Image Super-Resolution, Bee Lim, 2017
• RNN
• Long Short-Term Memory (LSTM), Jürgen Schmidhuber, Sepp Hochreiter, 1997
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation (GRU), Kyunghyun Cho, 2014
• Adaptive Computation Time for Recurrent Neural Networks (ACT), Alex Graves, 2016
• LSTM: A Search Space Odyssey, Klaus Greff, Jürgen Schmidhuber, 2017
• Dynamic Coattention Networks For Question Answering (DCN), Caiming Xiong, 2017
• Attention Is All You Need, Ashish Vaswani, 2017
• Computer Vision:
• Face Recognition
• General framework for object detection, Tomaso A. Poggio, 1998
• Rapid Object Detection using a Boosted Cascade of Simple Features, Viola & Jones, 2001
• Boosting Image Retrieval, K Tieu, 2004
• Object Recognition
• General framework for object detection, W Liu, 2015
• GANs
• Adaptive Deconvolutional Networks for Mid and High Level Feature Learning, MD Zeiler, 2011
• Generative Adversarial Nets, I Goodfellow, 2014
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Alec Radford, 2015
• Blog post: How do GANs intuitively work?, Chanchana Sornsoontorn, 2017
• Reinforcement:
• Using confidence bounds for exploitation-exploration trade-offs, Peter Auer, 2003
• Asynchronous Methods for Deep Reinforcement Learning, Volodymyr Mnih, 2016
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning, I Higgins, 2017
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation, Y Wu, 2017
• Distributional Reinforcement Learning with Quantile Regression, W Dabney, 2017
• CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning, C Colas, 2018
• Probabilistic Programming:
• Bayesian Networks
• Monte Carlo Markov Chains
• Metropolis–Hastings algorithm (wikipedia)
• Hamiltonian Markov Chains
• MCMC using Hamiltonian dynamics, Radford M. Neal, 2012
• A Conceptual Introduction to Hamiltonian Monte Carlo, Michael Betancourt, 2017
• Blog Post: Bayesian Inference, Markov Chain Monte Carlo and Metropolis Hastings, in python, Joseph Moukarzel, 2018
• NLP:
• Effective Approaches to Attention-based Neural Machine Translation, MT Luong, 2015
• seq2seq
• Sequence to Sequence Learning with Neural Networks, I Sutskever, 2014
• ULMFIT
• ELMo
• BERT
• BigBird
• MT DNN
• GPT-2

DATASETS

Images: MNIST, CIFAR-10, ImageNet, Breast Cancer Wisconsin, MS COCO, Open Images Dataset V4, COIL-100, Chars74K, Facial Image, Labelme, xView, Fashion-MNIST, PASCAL VOC, Street View House Numbers (SVHN), Stanford Dogs, Omniglot alphabets, VisualQA

Data: Iris Flowers, Wine Quality, Boston House Price

Videos: YouTube Dataset, CIFAR-10, ImageNet, Hollywood2, MovieLens, BDD100K

Corpus: WordNet, Twitter Sentiment Analysis, Sentiment140, BBC News, Amazon Reviews, Spam SMS Classifier, HotpotQA, Cornell Movie Dialogs, TIMIT Acoustic-Phonetic Continuous Speech, Reuters RCV1 RCV2 Multilingual, IMDB Reviews, WebQuestions

Lists of datasets: Wikipedia list of datasets, Kaggle datasets, Amazon datasets, Google datasets, Microsoft datasets, UCI datasets, Max Planck Institute datasets, Awesome Public Datasets, Computer Vision Datasets, EU Open Data Portal: European Government Datasets, Botanical Illustrations, Biodiversity data, ETH Zurich Robotics and Perception Group datasets, Lionbridge datasets, a blog post (Medium), another blog post (Analytics Vidhya)

TOOLS

Libraries (for python):
NumPy - scientific computing, arrays (cheatsheet)
SciPy - scientific computing (cheatsheet)
Pandas - data analysis, data structure (cheatsheet)
Scikit Learn - machine learning (cheatsheet)
Tensorflow - machine learning ecosystem (cheatsheet)
Keras - Neural Network API, python based, can run on top of tf, sequentials models (cheatsheet)
Pytorch - open source deep learning lib, based on C++ (cheatsheet)
MatPlotLib - plots and visualisations (cheatsheet)
Seaborn - plots and visualisations but nicer
NLTK - NLP text analysis most popular and useful lib
TextBlob - for esaier NLP and better than Pattern
SpaCy - for NLP with minimalism and selected options
Gensim - for well optimised and robust NLP
NetworkX - lib for graphs and networks Edward - lib for probabilistic modeling, inference, and criticism Pomegranate - lib for probabilistic models, bayes nets and Markov Models deeplearning.net - Software links

R / R Studio (cheatsheet)

Anaconda
Jupyter Notebook
Google Collab

AI cheatsheets

DeepIndex: what AI can do and where it is being applied

Pathmind, the Artificial Intelligence Wiki

Machine Learning Glossary: explanations of machine learning concepts with diagrams, code examples and links

Distill is dedicated to clear explanations of machine learning

Papers With Code: free and open resource with Machine Learning papers, code and evaluation tables(really REALLY cool state of the art)

Numerical Tours: some Jupyter Notebooks Python example

nteractive Machine Learning, Deep Learning and Statistics websites list

Lex Fridman (Artificial Intelligence podcast)

CONFERENCES

NeurIPS : Neural Information Processing Systems (NIPS)
ICML : International Conference on Machine Learning (ICML)
CVPR : IEEE/CVF Conference on Computer Vision and Pattern Recognition
ECAI : European Conference on Artificial Intelligence (ECAI)
ICRA
ICDL-EpiRob: International Conference on Development and Learning

BLOG POSTS

Machine Learning for Everyone. In simple words. With real-world examples. Yes, again by @vas3kcom
A Beginner's Guide To Understanding Convolutional Neural Networks by Adit Deshpande Artnome, Exploring Art Through Data Taxonomy vs Ontology: Machine Learning Breakthroughs A Short History of Ontology: It’s not just a Matter of Philosophy Anymore Cultures tribes of Machine Learning All Machine Learning Models Have Flaws

ML MEMES

AI Memes for Artificially Intelligent Teens (twitter)
deeplearning.ai (linkedin)
Machine Learning Memes for Convolutional Teens (facebook)
Mathematical Mathematics Memes (facebook)
Statistical Statistics Memes (facebook)

Cognitive Sciences

Mindmap of Cognitive Sciences Related Knowledge

BOOKS

LINKS

Le cerveau statisticien : la révolution Bayésienne en sciences cognitives, Stanislas Dehaene course at College de France

Allen Brain Atlas

Cognitive Bias Codex

Human Computer Interactions

BOOKS

ARTICLES

"Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms" by Hiroshi Ishii and Brygg Ullmer

"The Computer for the 21st Century" by Mark Weiser

Experimental study of apparent behavior, Fritz Heider & Marianne Simmel. 1944

CONFERENCES

CHI : CHI : Computer Human Interaction
SIGGRAPH : Special Interest Group on Computer GRAPHics and Interactive Techniques
UIST : ACM Symposium on User Interface Software and Technology
ICMI : International Conference on Multimodal Interfaces
VRST : ACM Symposium on Virtual Reality Software and Technology
ISS : International Conference on Interactive Surfaces and Spaces

LINKS

List of Physical Visualizations and Related Artifacts

List of funny bad designs

Can't Unsee

Maths, Stats and Bayesianism

BOOKS

ARTICLES

A Mathematical Theory of Communication, Claude Shannon, 1948

COOL THEOREMS

• Vectors:
• Matricies:
• Probability:
• Graphs:
• Multivariate Derivatives:
• Univariate Derivatives:
• Dynamical Systems:

LINKS

P-hacking: Hack Your Way To Scientific Glory

Spurious Correlations: Number of people who drowned by falling into a pool correlated with Films Nicolas Cage appeared in

Ockham Razor in one tweet

Significant "Jelly beans linked to acnea" (xkdc)

Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges (optimal decision making with bandits kinda solving the selection bias)

WolframAlpha Computational Engine

Brilliant (quizzes about math, logic and science)

Mathematical Tours: List of Algorithms

Youtube channels:
mathematicalmonk
Numberphile
3Blue1Brown
Science4All (in french)
Krista King
La statistique expliquée à mon chat (in french)

Programming, Code, Robotics

BOOKS

TOOLS

MongoDB - python document-oriented NoSQL database program (MongoDB Compass GUI)
Spark - for real time querying of data, has API for working with Big Data, permits parallel data processing
Spark MLib - ml algos, features, pipelines, etc
Flask- web framework
Django- python web framework
Scala- supports parallel data processing, compact, fast and efficient (processes everything in memory so it is fast)

LINKS

Teach Yourself Computer Science

Youtube channels:
Computerphile
Motivational advices video: "It's Dangerous to Go Alone: Battling the Invisible Monsters in Tech - PyCon 2014 (Julie Pagano)"