Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Python Data Analysis
  • Table Of Contents Toc
  • Feedback & Rating feedback
Python Data Analysis

Python Data Analysis

By : Ivan Idris
3.9 (16)
Buy this Book
close
close
Python Data Analysis

Python Data Analysis

3.9 (16)
By: Ivan Idris
Buy this Book

Overview of this book

This book is for programmers, scientists, and engineers who have knowledge of the Python language and know the basics of data science. It is for those who wish to learn different data analysis methods using Python and its libraries. This book contains all the basic ingredients you need to become an expert data analyst.
Table of Contents (22 chapters)
close
close
close
Python Data Analysis
Icon
Python Data Analysis
Credits
Icon
Credits
About the Author
Icon
About the Author
About the Reviewers
Icon
About the Reviewers
www.PacktPub.com
Icon
www.PacktPub.com
Preface
Icon
Preface
Free Chapter
1
Getting Started with Python Libraries
Icon
Getting Started with Python Libraries
Icon
Software used in this book
Icon
Building NumPy, SciPy, matplotlib, and IPython from source
Icon
Installing with setuptools
Icon
NumPy arrays
Icon
A simple application
Icon
Using IPython as a shell
Icon
Reading manual pages
Icon
IPython notebooks
Icon
Where to find help and references
Icon
Summary
2
NumPy Arrays
Icon
NumPy Arrays
Icon
The NumPy array object
Icon
Creating a multidimensional array
Icon
Selecting NumPy array elements
Icon
NumPy numerical types
Icon
One-dimensional slicing and indexing
Icon
Manipulating array shapes
Icon
Creating array views and copies
Icon
Fancy indexing
Icon
Indexing with a list of locations
Icon
Indexing NumPy arrays with Booleans
Icon
Broadcasting NumPy arrays
Icon
Summary
3
Statistics and Linear Algebra
Icon
Statistics and Linear Algebra
Icon
NumPy and SciPy modules
Icon
Basic descriptive statistics with NumPy
Icon
Linear algebra with NumPy
Icon
Finding eigenvalues and eigenvectors with NumPy
Icon
NumPy random numbers
Icon
Creating a NumPy-masked array
Icon
Summary
4
pandas Primer
Icon
pandas Primer
Icon
Installing and exploring pandas
Icon
pandas DataFrames
Icon
pandas Series
Icon
Querying data in pandas
Icon
Statistics with pandas DataFrames
Icon
Data aggregation with pandas DataFrames
Icon
Concatenating and appending DataFrames
Icon
Joining DataFrames
Icon
Handling missing values
Icon
Dealing with dates
Icon
Pivot tables
Icon
Remote data access
Icon
Summary
5
Retrieving, Processing, and Storing Data
chevron up
Icon
Retrieving, Processing, and Storing Data
Icon
Writing CSV files with NumPy and pandas
Icon
Comparing the NumPy .npy binary format and pickling pandas DataFrames
Icon
Storing data with PyTables
Icon
Reading and writing pandas DataFrames to HDF5 stores
Icon
Reading and writing to Excel with pandas
Icon
Using REST web services and JSON
Icon
Reading and writing JSON with pandas
Icon
Parsing RSS and Atom feeds
Icon
Parsing HTML with Beautiful Soup
Icon
Summary
6
Data Visualization
Icon
Data Visualization
Icon
matplotlib subpackages
Icon
Basic matplotlib plots
Icon
Logarithmic plots
Icon
Scatter plots
Icon
Legends and annotations
Icon
Three-dimensional plots
Icon
Plotting in pandas
Icon
Lag plots
Icon
Autocorrelation plots
Icon
Plot.ly
Icon
Summary
7
Signal Processing and Time Series
Icon
Signal Processing and Time Series
Icon
statsmodels subpackages
Icon
Moving averages
Icon
Window functions
Icon
Defining cointegration
Icon
Autocorrelation
Icon
Autoregressive models
Icon
ARMA models
Icon
Generating periodic signals
Icon
Fourier analysis
Icon
Spectral analysis
Icon
Filtering
Icon
Summary
8
Working with Databases
Icon
Working with Databases
Icon
Lightweight access with sqlite3
Icon
Accessing databases from pandas
Icon
SQLAlchemy
Icon
Pony ORM
Icon
Dataset – databases for lazy people
Icon
PyMongo and MongoDB
Icon
Storing data in Redis
Icon
Apache Cassandra
Icon
Summary
9
Analyzing Textual Data and Social Media
Icon
Analyzing Textual Data and Social Media
Icon
Installing NLTK
Icon
Filtering out stopwords, names, and numbers
Icon
The bag-of-words model
Icon
Analyzing word frequencies
Icon
Naive Bayes classification
Icon
Sentiment analysis
Icon
Creating word clouds
Icon
Social network analysis
Icon
Summary
10
Predictive Analytics and Machine Learning
Icon
Predictive Analytics and Machine Learning
Icon
A tour of scikit-learn
Icon
Preprocessing
Icon
Classification with logistic regression
Icon
Classification with support vector machines
Icon
Regression with ElasticNetCV
Icon
Support vector regression
Icon
Clustering with affinity propagation
Icon
Mean Shift
Icon
Genetic algorithms
Icon
Neural networks
Icon
Decision trees
Icon
Summary
11
Environments Outside the Python Ecosystem and Cloud Computing
Icon
Environments Outside the Python Ecosystem and Cloud Computing
Icon
Exchanging information with MATLAB/Octave
Icon
Installing rpy2
Icon
Interfacing with R
Icon
Sending NumPy arrays to Java
Icon
Integrating SWIG and NumPy
Icon
Integrating Boost and Python
Icon
Using Fortran code through f2py
Icon
Setting up Google App Engine
Icon
Running programs on PythonAnywhere
Icon
Working with Wakari
Icon
Summary
12
Performance Tuning, Profiling, and Concurrency
Icon
Performance Tuning, Profiling, and Concurrency
Icon
Profiling the code
Icon
Installing Cython
Icon
Calling C code
Icon
Creating a process pool with multiprocessing
Icon
Speeding up embarrassingly parallel for loops with Joblib
Icon
Comparing Bottleneck to NumPy functions
Icon
Performing MapReduce with Jug
Icon
Installing MPI for Python
Icon
IPython Parallel
Icon
Summary
Key Concepts
Icon
Key Concepts
Useful Functions
Icon
Useful Functions
Icon
matplotlib
Icon
NumPy
Icon
pandas
Icon
Scikit-learn
Icon
SciPy
Online Resources
Icon
Online Resources
Index
Icon
Index

Comparing the NumPy .npy binary format and pickling pandas DataFrames

Saving data in the CSV format is fine most of the time. It is easy to exchange CSV files, since most programming languages and applications can handle this format. However, it is not very efficient; CSV and other plaintext formats take up a lot of space. Numerous file formats have been invented, which offer a high level of compression such as zip, bzip, and gzip.

The following is the complete code for this storage comparison exercise, which can also be found in the binary_formats.py file of this book's code bundle:

import numpy as np
import pandas as pd
from tempfile import NamedTemporaryFile
from os.path import getsize

np.random.seed(42)
a = np.random.randn(365, 4)

tmpf = NamedTemporaryFile()
np.savetxt(tmpf, a, delimiter=',')
print "Size CSV file", getsize(tmpf.name)

tmpf = NamedTemporaryFile()
np.save(tmpf, a)
tmpf.seek(0)
loaded = np.load(tmpf)
print "Shape", loaded.shape
print "Size .npy file", getsize(tmpf.name...

Limited Time Offer

$10p/m for 3 months

Get online access to our library of over 7000 practical eBooks and videos, constantly updated with the latest in tech and supported with AI assistants
Claim Now Or Sign In
End of Section 3
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon

Create a Note

Modal Close icon
You need to login to use this feature.

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete

Delete Note

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete

Edit Note

Modal Close icon
Write a note (max 255 characters)
Cancel
Update Note