This is a templatized and heterogeneous C++ container with data-analysis functionality and interface.
template<typename I, typename H> class DataFrame;I specifies the index column type
template<typename I> using StdDataFrame = DataFrame<I, HeteroVector>; template<typename I> using DataFrameView = DataFrame<I, HeteroView>; template<typename I> using DataFramePtrView = DataFrame<I, HeteroPtrView>;
DataFrame Types |
---|
enum class box_cox_type{ } |
enum class bucket_type{ } |
enum class concat_policy{ } |
enum class decompose_type{ } |
enum class drop_policy{ } |
enum class fill_policy{ } |
enum class exponential_decay_spec{ } |
enum class hampel_type{ } |
enum class Index2D{ } |
enum class io_format{ } |
enum class join_policy{ } |
enum class mad_type{ } |
enum class mean_type{ } |
enum class nan_policy{ } |
enum class pattern_spec{ } |
enum class quantile_policy{ } |
enum class random_policy{ } |
enum class rank_policy{ } |
enum class remove_dup_spec{ } |
enum class return_policy{ } |
enum class shift_policy{ } |
enum class sigmoid_type{ } |
enum class sort_spec{ } |
enum class sort_state{ } |
enum class time_frequency{ } |
operator df_divides( ) |
operator df_minus( ) |
operator df_multiplies( ) |
operator df_plus( ) |
struct BadRange{ } |
struct ColNotFound{ } |
struct DataFrameError{ } |
struct InconsistentData{ } |
struct MemUsage{ } |
struct NotFeasible{ } |
struct NotImplemented{ } |
Stand-alone Numeric Generators |
---|
gen_bernoulli_dist{ } |
gen_binomial_dist( ) |
gen_cauchy_dist( ) |
gen_chi_squared_dist( ) |
gen_even_space_nums( ) |
gen_exponential_dist( ) |
gen_extreme_value_dist( ) |
gen_fisher_f_dist( ) |
gen_gamma_dist( ) |
gen_geometric_dist( ) |
gen_log_space_nums( ) |
gen_lognormal_dist( ) |
gen_negative_binomial_dist( ) |
gen_normal_dist( ) |
gen_poisson_dist( ) |
gen_student_t_dist( ) |
gen_sym_triangle( ) |
gen_triangular_nums( ) |
gen_uniform_int_dist( ) |
gen_uniform_real_dist( ) |
gen_weibull_dist( ) |
Views have useful and practical use-cases. A view is a slice of a DataFrame that is a reference to the original DataFrame. It appears exactly the same as a DataFrame, but if you modify any data in the view, the corresponding data point(s) in the original DataFrame will also be modified and vice versa. There are certain things you cannot do in views. For example, you cannot add or delete columns, extend the index column, ...
For more understanding, look at this document further and/or the test files.
Visitors are the main mechanism to implement analytical (i.e. statistical, financial, machine-learning) algorithms. You can easily follow the visitor's interface to add your custom algorithm by which you will extend the DataFrame package. Visitors also play several roles that in other packages maybe handled by separate interfaces. Visitors play the role of apply, transformer, and algorithms. For example, a visitors can transform column(s) or it may take the column(s) as read-only and implement an algorithm.
There are two visitor interfaces:
Random generators, and a few other numeric generators, were added as a series of convenient stand-alone functions to generate random numbers (it covers all C++ standard distributions). You can seamlessly use these routines to generate random DataFrame columns.
See this document and file RandGen.h and dataframe_tester.cc.
For the definition and defaults of RandGenParams, see this document and file DataFrameTypes.h
The DataFrame library is almost a header-only library with a few boilerplate source file exceptions, HeteroVector.cc and HeteroView.cc and a few others. Also, there is DateTime.cc.
Starting from the root directory:
include directory contains most of the code. It includes .h and .tcc files. The latter are C++ template code files (they are mostly located in the Internals subdirectory). The main header file is DataFrame.h. It contains the DataFrame class and its public interface. There are comprehensive comments for each public interface call in that file. The rest of the files will show you how the sausage is made. Include directory also contains subdirectories that contain mostly internal DataFrame implementation. One exception, the DateTime.h is located in the Utils subdirectory
src directory contains Linux-only make files and a few subdirectories that contain various source codes.
test directory contains all the test source files, mocked data files, and test output files. The main test source files are dataframe_tester.cc and dataframe_tester_2.cc. It contains test cases for all functionalities of DataFrame. It is not in a very organized structure. I plan to make the test cases more organized.
Using plain make and make-files:
Go to the root of the repository, where license file is, and execute build_all.sh. This will build the library and test executables for Linux/Unix flavors only
Using cmake:
Please see README file. Thanks to @justinjk007, you should be able to build this in Linux, Windows, Mac, and more
Although Pandas has a spot-on interface and it is full of useful functionalities, it lacks performance and scalability. For example, it is hard to decipher high-frequency intraday data such as Options data or S&P500 constituents tick-by-tick data using Pandas. Another issue I have encountered often is the research is done using Python, because it has such tools as Pandas, but the execution in production is in C++ for its efficiency, reliability and scalability. Therefore, there is this translation, or sometimes a bridge, between research and executions. Also, in this day and age, C++ needs a heterogeneous data container. Mainly because of these factors, I implemented the C++ DataFrame.
I welcome all contributions from people with expertise, interest, and time to do it. I will add more functionalities from time to time, but currently my spare time is limited.