raccoon package

Submodules

raccoon.dataframe module

DataFrame class

class raccoon.dataframe.DataFrame(data=None, columns=None, index=None, index_name='index', use_blist=False, sort=None)

Bases: object

DataFrame class. The raccoon DataFrame implements a simplified version of the pandas DataFrame with the key objective difference that the raccoon DataFrame is meant for use cases where the size of the DataFrame rows is expanding frequently. This is known to be slow with Pandas due to the use of numpy as the underlying data structure. Raccoon uses BList as the underlying data structure which is quick to expand and grow the size. The DataFrame can be designated as sort, in which case the rows will be sort by index on construction, and then any addition of a new row will insert it into the DataFrame so that the index remains sort.

Parameters:
  • data – (optional) dictionary of lists. The keys of the dictionary will be used for the column names and the lists will be used for the column data.
  • columns – (optional) list of column names that will define the order
  • index – (optional) list of index values. If None then the index will be integers starting with zero
  • index_name – (optional) name for the index. Default is “index”
  • use_blist – if True then use blist() as the underlying data structure, if False use standard list()
  • sort – if True then DataFrame will keep the index sort. If True all index values must be of same type
add(left_column, right_column, indexes=None)

Math helper method that adds element-wise two columns. If indexes are not None then will only perform the math on that sub-set of the columns.

Parameters:
  • left_column – first column name
  • right_column – second column name
  • indexes – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
Returns:

list

append(data_frame)

Append another DataFrame to this DataFrame. If the new data_frame has columns that are not in the current DataFrame then new columns will be created. All of the indexes in the data_frame must be different from the current indexes or will raise an error.

Parameters:data_frame – DataFrame to append
Returns:nothing
append_row(index, values, new_cols=True)

Appends a row of values to the end of the data. If there are new columns in the values and new_cols is True they will be added. Be very careful with this function as for sort DataFrames it will not enforce sort order. Use this only for speed when needed, be careful.

Parameters:
  • index – value of the index
  • values – dictionary of values
  • new_cols – if True add new columns in values, if False ignore
Returns:

nothing

append_rows(indexes, values, new_cols=True)

Appends rows of values to the end of the data. If there are new columns in the values and new_cols is True they will be added. Be very careful with this function as for sort DataFrames it will not enforce sort order. Use this only for speed when needed, be careful.

Parameters:
  • indexes – list of indexes
  • values – dictionary of values where the key is the column name and the value is a list
  • new_cols – if True add new columns in values, if False ignore
Returns:

nothing

blist
columns
data
delete_all_rows()

Deletes the contents of all rows in the DataFrame. This function is faster than delete_rows() to remove all information, and at the same time it keeps the container lists for the columns and index so if there is another object that references this DataFrame, like a ViewSeries, the reference remains in tact.

Returns:nothing
delete_columns(columns)

Delete columns from the DataFrame

Parameters:columns – list of columns to delete
Returns:nothing
delete_rows(indexes)

Delete rows from the DataFrame

Parameters:indexes – either a list of values or list of booleans for the rows to delete
Returns:nothing
divide(left_column, right_column, indexes=None)

Math helper method that divides element-wise two columns. If indexes are not None then will only perform the math on that sub-set of the columns.

Parameters:
  • left_column – column name of dividend
  • right_column – column name of divisor
  • indexes – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
Returns:

list

equality(column, indexes=None, value=None)

Math helper method. Given a column and optional indexes will return a list of booleans on the equality of the value for that index in the DataFrame to the value parameter.

Parameters:
  • column – column name to compare
  • indexes – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
  • value – value to compare
Returns:

list of booleans

classmethod from_json(json_string)

Creates and return a DataFrame from a JSON of the type created by to_json

Parameters:json_string – JSON
Returns:DataFrame
get(indexes=None, columns=None, as_list=False, as_dict=False)

Given indexes and columns will return a sub-set of the DataFrame. This method will direct to the below methods based on what types are passed in for the indexes and columns. The type of the return is determined by the types of the parameters.

Parameters:
  • indexes – index value, list of index values, or a list of booleans. If None then all indexes are used
  • columns – column name or list of column names. If None then all columns are used
  • as_list – if True then return the values as a list, if False return a DataFrame. This is only used if the get is for a single column
  • as_dict – if True then return the values as a dictionary, if False return a DataFrame. This is only used if the get is for a single row
Returns:

either DataFrame, list, dict or single value. The return is a shallow copy

get_cell(index, column)

For a single index and column value return the value of the cell

Parameters:
  • index – index value
  • column – column name
Returns:

value

get_columns(index, columns=None, as_dict=False)

For a single index and list of column names return a DataFrame of the values in that index as either a dict or a DataFrame

Parameters:
  • index – single index value
  • columns – list of column names
  • as_dict – if True then return the result as a dictionary
Returns:

DataFrame or dictionary

get_entire_column(column, as_list=False)

Shortcut method to retrieve a single column all rows. Since this is a common use case this method will be faster than the more general method.

Parameters:
  • column – single column name
  • as_list – if True return a list, if False return DataFrame
Returns:

DataFrame is as_list if False, a list if as_list is True

get_location(location, columns=None, as_dict=False, index=True)

For an index location and list of columns return a DataFrame of the values. This is optimized for speed because it does not need to lookup the index location with a search. Also can accept relative indexing from the end of the DataFrame in standard python notation [-3, -2, -1]

Parameters:
  • location – index location in standard python form of positive or negative number
  • columns – list of columns, or None to include all columns
  • as_dict – if True then return a dictionary
  • index – if True then include the index in the dictionary if as_dict=True
Returns:

DataFrame or dictionary

get_locations(locations, columns=None, **kwargs)

For list of locations and list of columns return a DataFrame of the values.

Parameters:
  • locations – list of index locations
  • columns – list of column names
  • kwargs – will pass along these parameters to the get() method
Returns:

DataFrame

get_matrix(indexes, columns)

For a list of indexes and list of columns return a DataFrame of the values.

Parameters:
  • indexes – either a list of index values or a list of booleans with same length as all indexes
  • columns – list of column names
Returns:

DataFrame

get_rows(indexes, column, as_list=False)

For a list of indexes and a single column name return the values of the indexes in that column.

Parameters:
  • indexes – either a list of index values or a list of booleans with same length as all indexes
  • column – single column name
  • as_list – if True return a list, if False return DataFrame
Returns:

DataFrame is as_list if False, a list if as_list is True

get_slice(start_index=None, stop_index=None, columns=None, as_dict=False)

For sorted DataFrames will return either a DataFrame or dict of all of the rows where the index is greater than or equal to the start_index if provided and less than or equal to the stop_index if provided. If either the start or stop index is None then will include from the first or last element, similar to standard python slide of [:5] or [:5]. Both end points are considered inclusive.

Parameters:
  • start_index – lowest index value to include, or None to start from the first row
  • stop_index – highest index value to include, or None to end at the last row
  • columns – list of column names to include, or None for all columns
  • as_dict – if True then return a tuple of (list of index, dict of column names: list data values)
Returns:

DataFrame or tuple

head(rows)

Return a DataFrame of the first N rows

Parameters:rows – number of rows
Returns:DataFrame
index

Return a view of the index as a list. Because this is a view any change to the return list from this method will corrupt the DataFrame.

Returns:list
index_name
isin(column, compare_list)

Returns a boolean list where each elements is whether that element in the column is in the compare_list.

Parameters:
  • column – single column name, does not work for multiple columns
  • compare_list – list of items to compare to
Returns:

list of booleans

iterrows(index=True)

Iterates over DataFrame rows as dictionary of the values. The index will be included.

Parameters:index – if True include the index in the results
Returns:dictionary
itertuples(index=True, name='Raccoon')

Iterates over DataFrame rows as tuple of the values.

Parameters:
  • index – if True then include the index
  • name – name of the namedtuple
Returns:

namedtuple

multiply(left_column, right_column, indexes=None)

Math helper method that multiplies element-wise two columns. If indexes are not None then will only perform the math on that sub-set of the columns.

Parameters:
  • left_column – first column name
  • right_column – second column name
  • indexes – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
Returns:

list

rename_columns(rename_dict)

Renames the columns

Parameters:rename_dict – dict where the keys are the current column names and the values are the new names
Returns:nothing
reset_index(drop=False)

Resets the index of the DataFrame to simple integer list and the index name to ‘index’. If drop is True then the existing index is dropped, if drop is False then the current index is made a column in the DataFrame with the index name the name of the column. If the index is a tuple multi-index then each element of the tuple is converted into a separate column. If the index name was ‘index’ then the column name will be ‘index_0’ to not conflict on print().

Parameters:drop – if True then the current index is dropped, if False then index converted to columns
Returns:nothing
select_index(compare, result='boolean')

Finds the elements in the index that match the compare parameter and returns either a list of the values that match, of a boolean list the length of the index with True to each index that matches. If the indexes are tuples then the compare is a tuple where None in any field of the tuple will be treated as “*” and match all values.

Parameters:
  • compare – value to compare as a singleton or tuple
  • result – ‘boolean’ = returns a list of booleans, ‘value’ = returns a list of index values that match
Returns:

list of booleans or values

set(indexes=None, columns=None, values=None)

Given indexes and columns will set a sub-set of the DataFrame to the values provided. This method will direct to the below methods based on what types are passed in for the indexes and columns. If the indexes or columns contains values not in the DataFrame then new rows or columns will be added.

Parameters:
  • indexes – indexes value, list of indexes values, or a list of booleans. If None then all indexes are used
  • columns – columns name, if None then all columns are used. Currently can only handle a single column or all columns
  • values – value or list of values to set (index, column) to. If setting just a single row, then must be a dict where the keys are the column names. If a list then must be the same length as the indexes parameter, if indexes=None, then must be the same and length of DataFrame
Returns:

nothing

set_cell(index, column, value)

Sets the value of a single cell. If the index and/or column is not in the current index/columns then a new index and/or column will be created.

Parameters:
  • index – index value
  • column – column name
  • value – value to set
Returns:

nothing

set_column(index=None, column=None, values=None)

Set a column to a single value or list of values. If any of the index values are not in the current indexes then a new row will be created.

Parameters:
  • index – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
  • column – column name
  • values – either a single value or a list. The list must be the same length as the index list if the index list is values, or the length of the True values in the index list if the index list is booleans
Returns:

nothing

set_location(location, values, missing_to_none=False)

Sets the column values, as given by the keys of the values dict, for the row at location to the values of the values dict. If missing_to_none is False then columns not in the values dict will be left unchanged, if it is True then they are set to None. This method does not add new columns and raises an error.

Parameters:
  • location – location
  • values – dict of column names as keys and the value as the value to set the row for that column to
  • missing_to_none – if True set any column missing in the values to None, otherwise leave unchanged
Returns:

nothing

set_locations(locations, column, values)

For a list of locations and a column set the values.

Parameters:
  • locations – list of index locations
  • column – column name
  • values – list of values or a single value
Returns:

nothing

set_row(index, values)

Sets the values of the columns in a single row.

Parameters:
  • index – index value
  • values – dict with the keys as the column names and the values what to set that column to
Returns:

nothing

show(index=True, **kwargs)

Print the contents of the DataFrame. This method uses the tabulate function from the tabulate package. Use the kwargs to pass along any arguments to the tabulate function.

Parameters:
  • index – If True then include the indexes as a column in the output, if False ignore the index
  • kwargs – Parameters to pass along to the tabulate function
Returns:

output of the tabulate function

sort
sort_columns(column, key=None, reverse=False)

Sort the DataFrame by one of the columns. The sort modifies the DataFrame inplace. The key and reverse parameters have the same meaning as for the built-in sort() function.

Parameters:
  • column – column name to use for the sort
  • key – if not None then a function of one argument that is used to extract a comparison key from each list element
  • reverse – if True then the list elements are sort as if each comparison were reversed.
Returns:

nothing

sort_index()

Sort the DataFrame by the index. The sort modifies the DataFrame inplace

Returns:nothing
subtract(left_column, right_column, indexes=None)

Math helper method that subtracts element-wise two columns. If indexes are not None then will only perform the math on that sub-set of the columns.

Parameters:
  • left_column – first column name
  • right_column – name of column to subtract from the left_column
  • indexes – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
Returns:

list

tail(rows)

Return a DataFrame of the last N rows

Parameters:rows – number of rows
Returns:DataFrame
to_dict(index=True, ordered=False)

Returns a dict where the keys are the column names and the values are lists of the values for that column.

Parameters:
  • index – If True then include the index in the dict with the index_name as the key
  • ordered – If True then return an OrderedDict() to preserve the order of the columns in the DataFrame
Returns:

dict or OrderedDict()

to_json()

Returns a JSON of the entire DataFrame that can be reconstructed back with raccoon.from_json(input). Any object that cannot be serialized will be replaced with the representation of the object using repr(). In that instance the DataFrame will have a string representation in place of the object and will not reconstruct exactly.

Returns:json string
to_list()

For a single column DataFrame returns a list of the values. Raises error if more then one column.

Returns:list
validate_integrity()

Validate the integrity of the DataFrame. This checks that the indexes, column names and internal data are not corrupted. Will raise an error if there is a problem.

Returns:nothing

raccoon.series module

Series class

class raccoon.series.Series(data=None, index=None, data_name='value', index_name='index', use_blist=False, sort=None)

Bases: raccoon.series.SeriesBase

Series class. The raccoon Series implements a simplified version of the pandas Series with the key objective difference that the raccoon Series is meant for use cases where the size of the Series is expanding frequently. This is known to be slow with Pandas due to the use of numpy as the underlying data structure. The Series can be designated as sort, in which case the rows will be sort by index on construction, and then any addition of a new row will insert it into the Series so that the index remains sort.

Parameters:
  • data – (optional) list of values.
  • index – (optional) list of index values. If None then the index will be integers starting with zero
  • data_name – (optional) name of the data column, or will default to ‘value’
  • index_name – (optional) name for the index. Default is “index”
  • use_blist – if True then use blist() as the underlying data structure, if False use standard list()
  • sort – if True then Series will keep the index sort. If True all index values must be of same type
append_row(index, value)

Appends a row of value to the end of the data. Be very careful with this function as for sorted Series it will not enforce sort order. Use this only for speed when needed, be careful.

Parameters:
  • index – index
  • value – value
Returns:

nothing

append_rows(indexes, values)

Appends values to the end of the data. Be very careful with this function as for sort DataFrames it will not enforce sort order. Use this only for speed when needed, be careful.

Parameters:
  • indexes – list of indexes to append
  • values – list of values to append
Returns:

nothing

blist
data
delete(indexes)

Delete rows from the DataFrame

Parameters:indexes – either a list of values or list of booleans for the rows to delete
Returns:nothing
index
reset_index()

Resets the index of the Series to simple integer list and the index name to ‘index’.

Returns:nothing
set(indexes, values=None)

Given indexes will set a sub-set of the Series to the values provided. This method will direct to the below methods based on what types are passed in for the indexes. If the indexes contains values not in the Series then new rows or columns will be added.

Parameters:
  • indexes – indexes value, list of indexes values, or a list of booleans.
  • values – value or list of values to set. If a list then must be the same length as the indexes parameter.
Returns:

nothing

set_cell(index, value)

Sets the value of a single cell. If the index is not in the current index then a new index will be created.

Parameters:
  • index – index value
  • value – value to set
Returns:

nothing

set_location(location, value)

For a location set the value

Parameters:
  • location – location
  • value – value
Returns:

nothing

set_locations(locations, values)

For a list of locations set the values.

Parameters:
  • locations – list of index locations
  • values – list of values or a single value
Returns:

nothing

set_rows(index, values=None)

Set rows to a single value or list of values. If any of the index values are not in the current indexes then a new row will be created.

Parameters:
  • index – list of index values or list of booleans. If a list of booleans then the list must be the same length as the Series
  • values – either a single value or a list. The list must be the same length as the index list if the index list is values, or the length of the True values in the index list if the index list is booleans
Returns:

nothing

sort
sort_index()

Sort the Series by the index. The sort modifies the Series inplace

Returns:nothing
class raccoon.series.SeriesBase

Bases: object

Base Series abstract base class that concrete implementations inherit from. Note that the .data and .index property methods in Series are views to the underlying data and not copies.

No specific parameters, those are defined in the child classed

data
data_name
equality(indexes=None, value=None)

Math helper method. Given a column and optional indexes will return a list of booleans on the equality of the value for that index in the DataFrame to the value parameter.

Parameters:
  • indexes – list of index values or list of booleans. If a list of booleans then the list must be the same length as the DataFrame
  • value – value to compare
Returns:

list of booleans

get(indexes, as_list=False)

Given indexes will return a sub-set of the Series. This method will direct to the specific methods based on what types are passed in for the indexes. The type of the return is determined by the types of the parameters.

Parameters:
  • indexes – index value, list of index values, or a list of booleans.
  • as_list – if True then return the values as a list, if False return a Series.
Returns:

either Series, list, or single value. The return is a shallow copy

get_cell(index)

For a single index and return the value

Parameters:index – index value
Returns:value
get_location(location)

For an index location return a dict of the index and value. This is optimized for speed because it does not need to lookup the index location with a search. Also can accept relative indexing from the end of the SEries in standard python notation [-3, -2, -1]

Parameters:location – index location in standard python form of positive or negative number
Returns:dictionary
get_locations(locations, as_list=False)

For list of locations return a Series or list of the values.

Parameters:
  • locations – list of index locations
  • as_list – True to return a list of values
Returns:

Series or list

get_rows(indexes, as_list=False)

For a list of indexes return the values of the indexes in that column.

Parameters:
  • indexes – either a list of index values or a list of booleans with same length as all indexes
  • as_list – if True return a list, if False return Series
Returns:

Series if as_list if False, a list if as_list is True

get_slice(start_index=None, stop_index=None, as_list=False)

For sorted Series will return either a Series or list of all of the rows where the index is greater than or equal to the start_index if provided and less than or equal to the stop_index if provided. If either the start or stop index is None then will include from the first or last element, similar to standard python slide of [:5] or [:5]. Both end points are considered inclusive.

Parameters:
  • start_index – lowest index value to include, or None to start from the first row
  • stop_index – highest index value to include, or None to end at the last row
  • as_list – if True then return a list of the indexes and values
Returns:

Series or tuple of (index list, values list)

head(rows)

Return a Series of the first N rows

Parameters:rows – number of rows
Returns:Series
index
index_name
isin(compare_list)

Returns a boolean list where each elements is whether that element in the column is in the compare_list.

Parameters:compare_list – list of items to compare to
Returns:list of booleans
select_index(compare, result='boolean')

Finds the elements in the index that match the compare parameter and returns either a list of the values that match, of a boolean list the length of the index with True to each index that matches. If the indexes are tuples then the compare is a tuple where None in any field of the tuple will be treated as “*” and match all values.

Parameters:
  • compare – value to compare as a singleton or tuple
  • result – ‘boolean’ = returns a list of booleans, ‘value’ = returns a list of index values that match
Returns:

list of booleans or values

show(index=True, **kwargs)

Print the contents of the Series. This method uses the tabulate function from the tabulate package. Use the kwargs to pass along any arguments to the tabulate function.

Parameters:
  • index – If True then include the indexes as a column in the output, if False ignore the index
  • kwargs – Parameters to pass along to the tabulate function
Returns:

output of the tabulate function

sort
tail(rows)

Return a Series of the last N rows

Parameters:rows – number of rows
Returns:Series
to_dict(index=True, ordered=False)

Returns a dict where the keys are the data and index names and the values are list of the data and index.

Parameters:
  • index – If True then include the index in the dict with the index_name as the key
  • ordered – If True then return an OrderedDict() to preserve the order of the columns in the Series
Returns:

dict or OrderedDict()

validate_integrity()

Validate the integrity of the Series. This checks that the indexes, column names and internal data are not corrupted. Will raise an error if there is a problem.

Returns:nothing
class raccoon.series.ViewSeries(data=None, index=None, data_name='value', index_name='index', sort=False, offset=0)

Bases: raccoon.series.SeriesBase

ViewSeries class. The raccoon ViewSeries implements a view only version of the Series object with the key objective difference that the raccoon ViewSeries is meant for view only use cases where the underlying index and data are modified elsewhere or static. Use this for a view into a single column of a DataFrame.

Parameters:
  • data – (optional) list of values.
  • index – (optional) list of index values. If None then the index will be integers starting with zero
  • data_name – (optional) name of the data column, or will default to ‘value’
  • index_name – (optional) name for the index. Default is “index”
  • sort – if True then assumes the index is sorted for faster set/get operations
  • offset – integer to add to location to transform to standard python list location index
data
classmethod from_dataframe(dataframe, column, offset=0)

Creates and return a Series from a DataFrame and specific column

Parameters:
  • dataframe – raccoon DataFrame
  • column – column name
  • offset – offset value must be provided as there is no equivalent for a DataFrame
Returns:

Series

classmethod from_series(series, offset=0)

Creates and return a Series from a Series

Parameters:
  • series – raccoon Series
  • offset – offset value must be provided as there is no equivalent for a DataFrame
Returns:

Series

index
offset
sort
value(indexes, int_as_index=False)

Wrapper function for get. It will return a list, no index. If the indexes are integers it will be assumed that they are locations unless int_as_index = True. If the indexes are locations then they will be rotated to the left by offset number of locations.

Parameters:
  • indexes – integer location, single index, list of indexes or list of boolean
  • int_as_index – if True then will treat int index values as indexes and not locations
Returns:

value or list of values

raccoon.sort_utils module

Utility functions for sorting and dealing with sorted Series and DataFrames

raccoon.sort_utils.sorted_exists(values, x)

For list, values, returns the insert position for item x and whether the item already exists in the list. This allows one function call to return either the index to overwrite an existing value in the list, or the index to insert a new item in the list and keep the list in sorted order.

Parameters:
  • values – list
  • x – item
Returns:

(exists, index) tuple

raccoon.sort_utils.sorted_index(values, x)

For list, values, returns the index location of element x. If x does not exist will raise an error.

Parameters:
  • values – list
  • x – item
Returns:

integer index

raccoon.sort_utils.sorted_list_indexes(list_to_sort, key=None, reverse=False)

Sorts a list but returns the order of the index values of the list for the sort and not the values themselves. For example is the list provided is [‘b’, ‘a’, ‘c’] then the result will be [2, 1, 3]

Parameters:
  • list_to_sort – list to sort
  • key – if not None then a function of one argument that is used to extract a comparison key from each list element
  • reverse – if True then the list elements are sorted as if each comparison were reversed.
Returns:

list of sorted index values

raccoon.utils module

Raccoon utilities

raccoon.utils.assert_frame_equal(left, right, data_function=None, data_args=None)

For unit testing equality of two DataFrames.

Parameters:
  • left – first DataFrame
  • right – second DataFrame
  • data_function – if provided will use this function to assert compare the df.data
  • data_args – arguments to pass to the data_function
Returns:

nothing

raccoon.utils.assert_series_equal(left, right, data_function=None, data_args=None)

For unit testing equality of two Series.

Parameters:
  • left – first Series
  • right – second Series
  • data_function – if provided will use this function to assert compare the df.data
  • data_args – arguments to pass to the data_function
Returns:

nothing

Module contents