Data Science Summer School

Overview and Objectives

Data Science Summer School Week 1 and 2 participants are expected to have a basic knowledge of linear algebra, basic computing skills, and familiarity with any kind of programming language.

However, Deep Learning and Applications in the Week 3 is taught at a higher level and participants must know fundamentals of programming and data analysis with Python and fundamental concepts in statistical learning, overall the type of material which is covered in Week 1 of the Data Science Summer School (Foundations of Data Science).

Course Outline

In this course we will introduce several aspects of modern machine learning, deep learning and it’s applications:

1. An overall introduction to Deep Learning covering the fundamental architectures such as convolutional and recurrent natural networks, auto-encoders, as well as the basic optimization algorithms for training deep models.
2. Three individual, largely practical, sessions which cover different paradigms or application domains:

2.1. Computer Vision, with focus on advanced convolutional network architectures and generative models for tasks such as image classification, object detection or image captioning.

2.2. Natural Language Processing, with focus on several aspects of language modeling, word and document embeddings, conversational models and visualization.

2.3. Deep Reinforcement Learning, with focus on how deep learning can be used in sequential decision making problems such as board/video-games.

In case candidates for this course are not also taking a Week 1 course, they must upload a statement of purpose explaining their qualification for the course at the moment of applying. The statement should include any relevant info about academic knowledge and personal skills and interests as well as professional experience in the sector if any. In case of having joined the Data Science Summer School in the past, this should be mentioned in the statement.

About the Instructors

Vicenç Gómez is currently a tenure-track faculty under a Ramon y Cajal fellowship in the Artificial Intelligence and Machine Learning group at UPF, which he joined with a transnational academic career grant (FP7 Marie Curie Actions) in 2014. Prior to this, he worked for more than six years as a research scientist in the machine learning group at the Radboud University Nijmegen, the Netherlands. His main research interests are probabilistic inference and reinforcement learning. He works on developing novel machine learning methods derived from first principles and understanding their theoretical properties, as well as their application for modeling, understanding and improving the functioning of networked systems.

Anders Jonsson is professor in the ICT Department of UPF, and director of the Artificial Intelligence and Machine Learning group. He received his MSc in Engineering Physics from the Royal Institute of Technology in Stockholm, Sweden, and his PhD in Computer Science from the University of Massachusetts Amherst, USA. His research work is mainly focused on sequential decision problems, either in the form of reinforcement learning, or in the form of automatic planning. In particular, his work focuses on finding and exploiting the structure of these types of problems in order to simplify their solution. He has participated in numerous international projects.

Reading Material:

1. Deep Learning for Coders with fastai and PyTorch: AI Applications Without a PhD. Jeremy Howard, Sylvain Gugger. Publisher: O'Reilly Media

2. Representation Learning: A Review and New Perspectives. Yoshua Bengio, Aaron Courville, Pascal Vincent

3. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. Song Han, Huizi Mao, William J. Dally

4. Distilling the Knowledge in a Neural Network - Geoffrey Hinton, Oriol Vinyals, Jeff Dean

5. Pytorch tutorials

Prerequisites to Enroll

Basic Python and R knowledge are mandatory in order to follow this course. 

To check your own skill level before you apply, download the self-evaluation below and make sure you can solve the problem presented with both Python and R. (Solving with only one of the languages is not sufficient to be able to follow this course.)

Download and complete the self-evaluation before you apply

This exercise is not part of the application process. It will not be reviewed by the BSE admissions team or the instructors. For the interest of all participants, a proposed solution will be shown in class on the first day of the course.

Some knowledge of the following topics is also recommended:

• Jupyter notebooks
• Algebra

Students must have their own laptop or desktop computer, and a good internet connection. The latest version of Anaconda with Python and R should be installed, following instructions to be sent. Check hardware requirements to run Anaconda here.

Notebooks might be run as well using Google Collaboratory.

Further instructions will be sent prior to the start of the course.

Overview and Objectives

This is an intensive 20-hour course based on a hands-on approach using Jupyter notebooks, all material is motivated by specific information retrieval and data analysis questions and each thematic unit concludes with a small project. The course provides basic training in data analysis and machine learning with Python and R.

Required Activities 

Students will have access to videos corresponding to most of the topics, where the notebooks and materials are explained in advance. It is crucial to watch them in advance so that we can speed up during online sessions, focus on Q&A and solve the short exercises that are embedded in the notebooks. The estimated preparation time for those videos is around 25-30h. You will be give access to this videos before the course starts.

Course Outline

Classes last 5 days, 4 hours per day

The course provides training in data analysis and machine learning with Python and R and evolves along the thematic following units

1. Data analysis with Python (20’)

• Hands-on review of main methods in Python to explore data and gain insights for a particular assignment.

Keywords: data types, objects, pandas

2. Data visualization (1h30’)

• Elements of data visualization
• Scatter plots
• Line plots
• Exploration plots: barplots, boxplots
• Advanced plots: correlation, regression, biplots
• Special plots
• Reporting using visualization

Keywords: seaborn, plotly

3. Data preparation (2h)

• Handling missing data: imputation methods
• Feature transformation and engineering: normalization, dimensionality reduction, category encoding

Keywords: sklearn

4. Supervised learning (7h30’)

• General structure of a predictive model
  • Regression
  • Classification
• Linear models for regression
  • Linear models and non-linear feature maps
  • Model evaluation
  • Convexity
  • Bias-Variance tradeoff
  • Penalized likelihood and lasso
  • Cross validation and model selection
• Linear models for classification
  • Logistic regression
  • Misclassification, ROC, AUC
  • Class imbalance
  • Generative vs. discriminative models
• Non-linear models: decision trees
  • Decision trees
  • Variable selection
  • Forests 
  • Bagging and boosting

Keywords: sklearn, linear models, cross validation, regularization, lasso, trees, ensembles, boosting

5. Unsupervised learning (3h30’)

• Sample unsupervised project (clustering) in Python
• Continuous latent variables
• PCA and SVD
• Multidimensional Scaling
• Clustering

Keywords: clustering, factors, independent component analysis

6. Data analysis with R (20’)

• Hands-on review of main methods in R to explore data and gain insights for a particular assignment.

Keywords: data types, objects, dplyr

7. Intro to data science in R (4h30’)

Same approach as for the Python syllabus but condensed. Student will be able to map Python version of learning to a very similar R version of same notebooks.

• Data visualization in R
  • Scatter plots
  • Line plots
  • Exploration plots
  • Advanced plots
• Sample predictive model in R
  • Regression
  • Classification
• Supervised and unsupervised learning with R
  • Supervised learning: linear, non-linear
  • Unsupervised learning: PCA, clustering

Keywords: tidyr, data.table, caret, ggplot2, plotly

Required Activities

Class participation is compulsory. During classes there will be little projects that students are expected to perform while in class.

Students will have access to videos corresponding to most of the topics, where the notebooks and materials are explained in advance. It is crucial to watch them in advance, so that we can speed up during online sessions, focus on Q&A and solve the short exercises that are embedded in the notebooks.

Each day there will be a project assignment, to be done individually, with some support from TAs (Teaching Assistants). This implies that, to follow the course contents, besides the regular classes, students would need to spend 3-4 hours each day to work on the assignments.

About the Instructors

The course will be delivered by André B.M. Souza, PhD candidate in the Department of Economics and Business at Universitat Pompeu Fabra. His research interests include forecasting, financial econometrics, machine learning, empirical finance and empirical macroeconomics. Throughout his PhD, he has taught several econometrics and statistics courses for the Master's programs at the BSE. In September 2021, André will join the department of Economics, Finance and Accounting at ESADE Business School as an Assistant Professor.

Overview and Objectives

The participants of this course are expected to be familiar with the fundamentals of programming, data analysis with Python and R as well as fundamental concepts in statistical learning, overall the type of material which is covered in the "Foundations of Data Science" course.

An ever-increasing share of human communication is recorded as digital text. Analyzing and making sense of this vast amount of data is increasingly important for research in the social sciences.

This course provides an accelerated introduction to the theory and practice of text analysis by surveying methods for systematically extracting quantitative information from text, from classical content analysis and dictionary-based methods, to classification methods, scaling methods, and topic models.

The course introduces the theoretical foundations for text analysis but mainly takes a practical approach, illustrating the methods through state-of-the-art applications to research questions in economics, political science, and finance. Lectures will cover a series of case-studies from economics and related fields, and will be complemented by hands-on programming sessions in Python.

By the end of the course students will be able to:

i) Convert vast archives of text into a format that can be used for data analysis

ii) Use the data generated from text to tackle research and policy problems relevant to their interests and organizations.

Course schedule

1. From text to data

• Introduction to documents, metadata, corpora
• Word counts, document-feature matrix
• Collocation and n-grams

2. Important notions

• TF-IDF
• Measuring text length, diversity, and complexity
• Measuring similarity between documents

3. Machine Learning

• Introduction to statistical learning theory
• Supervised and unsupervised learning

4. Statistical methods

• Dictionary-based methods
• Penalized linear models
• Dimension reduction and feature selection
• Non-Linear text regression
• Random forests

5. Generative language models: unsupervised methods

• Latent semantic analysis
• Topic models and Latent Dirichlet Allocation (LDA)
• K-means clustering

6. Word Embeddings

• Word2Vec
• Doc2Vec

About the instructors

Ruben Durante is ICREA Research Professor at UPF and Affiliated Professor of the BSE. He works in the field of political economics, with a focus on the functioning and impact of traditional and new media in democratic societies. His work has been published in a number of top economic journals, including the Journal of Political Economy, the American Economic Journal: Applied Economics, and the Journal of the European Economic Association, and has been featured extensively in the popular press.

Hannes Mueller is a tenured researcher at the Institute for Economic Analysis (IAE-CSIC) and an Associate Research Professor at the BSE. His fields of interest are Political Economy, Development Economics and Conflict Studies with a particular focus on the effect of violent conflict on the economy. Most recently, Prof. Mueller is trying to adopt supervised and unsupervised machine learning techniques for economics and political science research. He has published in leading journals in Economics and Political Science such as the American Economic Review (AER), the American Political Science Review (APSR), the Journal of the European Economic Association (JEEA) and the American Journal of Economics: Macro (AEJ: Macro). He has contributed reports for the International Growth Centre (UK government) and the World Bank on the economic effects of conflict, a joint UN/World Bank study on conflict prevention and the UN Economic Commission for Africa on structural change in Northern Africa. He is currently involved in projects with the Banco de España developing techniques for nowcasting and forecasting economic conditions with text.

Main readings:

• Bengfort, B., Ojeda, T., Bilbro, R., Applied Text Analysis with Python, 2018, O’Reilly Media.
• Gentzkow, M., Kelly, B. T., and Taddy, M., 2017, Text as Data. NBER Working Paper #23276.
• Grimmer, J. and Stewart, B., 2013, Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts, Political Analysis, vol. 21, n. 3, pp. 267-297.
• Krippendorff, K., 2013, Content Analysis: An Introduction to Its Methodology, Sage.
• Manning, C. D., Raghavan, P., and Shutze, H., 2008, An Introduction to Information Retrieval, Cambridge University Press.

Applications:

• Ash, E., Chen, D., and Naidu, S., 2018, Ideas Have Consequences: The Impact of Law and Economics on American Justice, Working Paper.
• Baker, S. R., Bloom, N., and Davis, S. J., 2016, Measuring Economic Policy Uncertainty, Quarterly Journal of Economics, vol. 131, n.4, pp. 1593-1636.
• Bandiera, O., Prat, A., Hansen, S., and Sadun, R., 2020, CEO Behavior and Firm Performance, Journal of Political Economy, vol. 128, n. 4, pp. 1325-1369.
• Blei, D., Ng, A., and Jordan, M., 2003, Latent Dirichlet Allocation, Journal of Machine Learning Research, vol. 3, pp. 993–1022.
• Blei, D., and Lafferty, D., 2006, Dynamic Topic Models, In Proceedings of the 23rd International Conference on Machine Learning.
• Cage, J., Hervé, N., and Viaud, M.-L., The Production of Information in an Online World, forthcoming, Review of Economic Studies.
• Garg, N., Schiebinger, L., Jurafsky, D., and Zou, J, 2018, Word embeddings quantify 100 years of gender and ethnic stereotypes, Proceedings of the National Academy of Sciences, vol. 115, n. 16, pp. 3635-3644
• Gentzkow, M. and Shapiro, J., 2009, What Drives Media Slant?, Econometrica, vol. 78, n.1, pp.35-71.
• Hassan, T., Hollander, S., van Lent, L., and Tahoun, A., 2018, Firm-Level Political Risk: Measurement and Effects, Working Paper.
• Hansen, S., McMahon, M., and Prat, A., 2018, Transparency and Deliberation within the FOMC: a Computational Linguistics Approach, Quarterly Journal of Economics, vol. 133, n. 2, pp. 801-870.
• Kelly, Bryan T., Papanikolaou, D., Seru, A., and Taddy, M, Measuring Technological Innovation Over the Long Run, forthcoming, American Economic Review: Insights.
• Loughran, T. and McDonald, B., 2011, When is a Liability not a Liability? Textual Analysis, Dictionaries, and 10K-s, The Journal of Finance, vol. 66, n. 1, pp. 35-65.
• Tetlock, P. C., 2007, Giving Content to Investor Sentiment: The Role of Media in the Stock Market, Journal of Finance, vol. 62, n. 3, pp. 1139-1168.