The Expertmaker Accelerator is a well-proven data processing framework that provides fast data access, parallel execution, and automatic organization of source code, input data, and results. It can be used for daily data analysis tasks as well as operating as a live recommendation system with hundreds of thousands of large data files and many CPUs. The Accelerator is released as open source by eBay.
The Accelerator projects has been built, tested, and runs on:
but is in no way limited to these systems or versions.
Install dependencies. On Debian and Ubuntu
sudo apt-get install build-essential python-dev python3-dev zlib1g-dev git virtualenv
cd accelerator-project_skeleton ./init.py
Please read and modify this script according to your needs.
The init.py script will clone both the accelerator-gzutil and the main accelerator repositories. The gzutil library will be set up in virtual environments for Python2 as well as Python3, and the Accelerator will be set up as a git submodule to the project_skeleton repository.
Copyright 2017-2018 eBay Inc.
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.