GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → CNevd → Difacto_dmlc

CNevd / Difacto_dmlc

Distributed FM and LR based on Parameter Server with Ftrl

Labels

distributed

Projects that are alternatives of or similar to Difacto dmlc

Fas
C Pixels-based graphical audio synthesizer implemented as a WebSocket server
Stars: ✭ 100 (-20.63%)
Mutual labels:  distributed
Distributed Dataset
A distributed data processing framework in Haskell.
Stars: ✭ 108 (-14.29%)
Mutual labels:  distributed
Vertex
Vertex is a distributed, ultimately consistent, event traceable cross platform framework based on Orleans, which is used to build high-performance, high throughput, low latency, scalable distributed applications
Stars: ✭ 117 (-7.14%)
Mutual labels:  distributed
Dandelion
a diaspora* client for Android
Stars: ✭ 100 (-20.63%)
Mutual labels:  distributed
Ipfs.ink
PROJECT HAS BEEN SHUTDOWN - Publish and render markdown essays to and from ipfs
Stars: ✭ 106 (-15.87%)
Mutual labels:  distributed
Raft.net
Implementation of RAFT distributed consensus algorithm among TCP Peers on .NET / .NETStandard / .NETCore / dotnet
Stars: ✭ 112 (-11.11%)
Mutual labels:  distributed
Turms
The world's most advanced open source instant messaging engine for 100K~10M concurrent users https://turms-im.github.io/docs
Stars: ✭ 97 (-23.02%)
Mutual labels:  distributed
Labgrid
embedded systems control library for development, testing and installation
Stars: ✭ 124 (-1.59%)
Mutual labels:  distributed
Micro
Micro is a distributed cloud operating system
Stars: ✭ 10,778 (+8453.97%)
Mutual labels:  distributed
Sandglass
Sandglass is a distributed, horizontally scalable, persistent, time sorted message queue.
Stars: ✭ 1,531 (+1115.08%)
Mutual labels:  distributed
Bojack
🐴 The unreliable key-value store
Stars: ✭ 101 (-19.84%)
Mutual labels:  distributed
Wpaxos
A production-grade java implementation of paxos consensus algorithm
Stars: ✭ 104 (-17.46%)
Mutual labels:  distributed
Xlearning Xdml
extremely distributed machine learning
Stars: ✭ 113 (-10.32%)
Mutual labels:  distributed
Foundatio
Pluggable foundation blocks for building distributed apps.
Stars: ✭ 1,365 (+983.33%)
Mutual labels:  distributed
Mangos V1
The pure golang implementation of nanomsg (version 1, frozen)
Stars: ✭ 1,533 (+1116.67%)
Mutual labels:  distributed
Nni
An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
Stars: ✭ 10,698 (+8390.48%)
Mutual labels:  distributed
Memo
The memo elastic and resilient key-value store.
Stars: ✭ 111 (-11.9%)
Mutual labels:  distributed
Hazelcast Nodejs Client
Hazelcast IMDG Node.js Client
Stars: ✭ 124 (-1.59%)
Mutual labels:  distributed
Orbit
A distributed, serverless, peer-to-peer chat application on IPFS
Stars: ✭ 1,586 (+1158.73%)
Mutual labels:  distributed
Xxl Sso
A distributed single-sign-on framework.（分布式单点登录框架XXL-SSO）
Stars: ✭ 1,635 (+1197.62%)
Mutual labels:  distributed
View All Similar Projects ➔

Distributed FM and LR with parameter server

GitHub license

Portable, scalable and reliable distributed machine learning.

Wormhole is a place where DMLC projects works together to provide scalable and reliable machine learning toolkits that can run on various platforms

Since wormhole has been Deprecated, we retain some useful tools and keep updating

Features

  • Portable:
    • Supported platforms: local machine, Apache YARN, MPI and Sungrid Engine
  • Rich support of Data Source
    • All projects can read data from HDFS, S3 or local filesystem
  • Scalable and Reliable

List of Tools

  • Linear method: linear
  • Factorization Machine based on PS: difacto

Build & Run

  • Requires a C++11 compiler (e.g.~g++ >=4.8) and git. Install them on Ubuntu >= 13.10

  • cd dmlc-core; make to make dmlc core
    cd ps-lite; make to make ps
    cd src/linear; make or cd src/difacto; make

Questions

  • How to set multipath?
    if you have date paths like: ./data/train1 ./data/train2
    pls set train_data = "./data/train1;./data/train2" or "./data/train.*"
    for hdfsfiles: train_data = "hdfs://data/train1;hdfs://data/train2" or "hdfs://data/train.*"
  • How to use HDFS?
    set USE_HDFS=1 in dmlc-core/make/config.mk and ps-lite/make/config.mk
  • How to get readable weight?
    use ./build/dump.dmlc model_in=your_model_path(should be local file) dump_out=dump_file need_inverse=1(0 or 1) then dump_file is the readable weight
  • dump error when use hdfs: ./build/dump.dmlc: error while loading shared libraries: libhdfs.so.0.0.0: cannot open shared object file: No such file or directory?
    pls add hadoop lib path to LD_LIBRARY_PATH before dump
    in my case: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/clusterserver/hadoop/lib/native/
  • Why the ids of dumped file is large like -2305843009213693952 and how to use the original ids?
    see issue8 and issue10
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].