Getting Started with Python Libraries | Python Data Analysis

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

Python Data Analysis

By : Ivan Idris

3.9 (16)

Python Data Analysis

3.9 (16)

By: Ivan Idris

Overview of this book

This book is for programmers, scientists, and engineers who have knowledge of the Python language and know the basics of data science. It is for those who wish to learn different data analysis methods using Python and its libraries. This book contains all the basic ingredients you need to become an expert data analyst.

Python Data Analysis

Python Data Analysis

Credits

Credits

About the Author

About the Author

About the Reviewers

About the Reviewers

www.PacktPub.com

www.PacktPub.com

Preface

Preface

Free Chapter

Getting Started with Python Libraries

Getting Started with Python Libraries

Software used in this book

Building NumPy, SciPy, matplotlib, and IPython from source

Installing with setuptools

NumPy arrays

A simple application

Using IPython as a shell

Reading manual pages

IPython notebooks

Where to find help and references

Summary

NumPy Arrays

NumPy Arrays

The NumPy array object

Creating a multidimensional array

Selecting NumPy array elements

NumPy numerical types

One-dimensional slicing and indexing

Manipulating array shapes

Creating array views and copies

Fancy indexing

Indexing with a list of locations

Indexing NumPy arrays with Booleans

Broadcasting NumPy arrays

Summary

Statistics and Linear Algebra

Statistics and Linear Algebra

NumPy and SciPy modules

Basic descriptive statistics with NumPy

Linear algebra with NumPy

Finding eigenvalues and eigenvectors with NumPy

NumPy random numbers

Creating a NumPy-masked array

Summary

pandas Primer

pandas Primer

Installing and exploring pandas

pandas DataFrames

pandas Series

Querying data in pandas

Statistics with pandas DataFrames

Data aggregation with pandas DataFrames

Concatenating and appending DataFrames

Joining DataFrames

Handling missing values

Dealing with dates

Pivot tables

Remote data access

Summary

Retrieving, Processing, and Storing Data

Retrieving, Processing, and Storing Data

Writing CSV files with NumPy and pandas

Comparing the NumPy .npy binary format and pickling pandas DataFrames

Storing data with PyTables

Reading and writing pandas DataFrames to HDF5 stores

Reading and writing to Excel with pandas

Using REST web services and JSON

Reading and writing JSON with pandas

Parsing RSS and Atom feeds

Parsing HTML with Beautiful Soup

Summary

Data Visualization

Data Visualization

matplotlib subpackages

Basic matplotlib plots

Logarithmic plots

Scatter plots

Legends and annotations

Three-dimensional plots

Plotting in pandas

Lag plots

Autocorrelation plots

Plot.ly

Summary

Signal Processing and Time Series

Signal Processing and Time Series

statsmodels subpackages

Moving averages

Window functions

Defining cointegration

Autocorrelation

Autoregressive models

ARMA models

Generating periodic signals

Fourier analysis

Spectral analysis

Filtering

Summary

Working with Databases

Working with Databases

Lightweight access with sqlite3

Accessing databases from pandas

SQLAlchemy

Pony ORM

Dataset – databases for lazy people

PyMongo and MongoDB

Storing data in Redis

Apache Cassandra

Summary

Analyzing Textual Data and Social Media

Analyzing Textual Data and Social Media

Installing NLTK

Filtering out stopwords, names, and numbers

The bag-of-words model

Analyzing word frequencies

Naive Bayes classification

Sentiment analysis

Creating word clouds

Social network analysis

Summary

Predictive Analytics and Machine Learning

Predictive Analytics and Machine Learning

A tour of scikit-learn

Preprocessing

Classification with logistic regression

Classification with support vector machines

Regression with ElasticNetCV

Support vector regression

Clustering with affinity propagation

Mean Shift

Genetic algorithms

Neural networks

Decision trees

Summary

Environments Outside the Python Ecosystem and Cloud Computing

Environments Outside the Python Ecosystem and Cloud Computing

Exchanging information with MATLAB/Octave

Installing rpy2

Interfacing with R

Sending NumPy arrays to Java

Integrating SWIG and NumPy

Integrating Boost and Python

Using Fortran code through f2py

Setting up Google App Engine

Running programs on PythonAnywhere

Working with Wakari

Summary

Performance Tuning, Profiling, and Concurrency

Performance Tuning, Profiling, and Concurrency

Profiling the code

Installing Cython

Calling C code

Creating a process pool with multiprocessing

Speeding up embarrassingly parallel for loops with Joblib

Comparing Bottleneck to NumPy functions

Performing MapReduce with Jug

Installing MPI for Python

IPython Parallel

Summary

Key Concepts

Key Concepts

Useful Functions

Useful Functions

matplotlib

NumPy

pandas

Scikit-learn

SciPy

Online Resources

Online Resources

Index

Index

Customer Reviews

3.9 (16)

5 star

37.5%

4 star

31.3%

3 star

18.8%

2 star

6.3%

1 star

6.3%

Building NumPy, SciPy, matplotlib, and IPython from source

As a last resort or if we want to have the latest code, we can build from source. In practice, it shouldn't be that hard, although depending on your operating system, you might run into problems. As operating systems and related software are rapidly evolving, in such cases, the best you can do is search online or ask for help. In this chapter, we give pointers on good places to look for help.

The source code can be retrieved with git or as an archive from GitHub. The steps to install NumPy from source are straightforward and given here. We can retrieve the source code for NumPy with git as follows:

$ git clone git://github.com/numpy/numpy.git numpy

Note

There are similar commands for SciPy, matplotlib, and IPython (refer to the table that follows after this piece of information). The IPython source code can be downloaded from https://github.com/ipython/ipython/releases as a source archive or ZIP file. You can then unpack it with your favorite tool or with the following command:

$ tar -xzf ipython.tar.gz

Please refer to the following table for the git commands and source archive/zip links:

Library	Git command	Tarball/zip URL
NumPy	git clone git://github.com/numpy/numpy.git numpy	https://github.com/numpy/numpy/releases
SciPy	git clone http://github.com/scipy/scipy.git scipy	https://github.com/scipy/scipy/releases
matplotlib	git clone git://github.com/matplotlib/matplotlib.git	https://github.com/matplotlib/matplotlib/releases
IPython	git clone --recursive https://github.com/ipython/ipython.git	https://github.com/ipython/ipython/releases

Install on /usr/local with the following command from the source code directory:

$ python setup.py build
$ sudo python setup.py install --prefix=/usr/local

To build, we need a C compiler such as GCC and the Python header files in the python-dev or python-devel package.

Search

Your notes and bookmarks