18dbaebebc
[Distributed] Specify the graph format for distributed training ( #2948 )
...
* explicitly set the graph format.
* fix.
* fix.
* fix launch script.
* fix readme.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
2021-05-26 18:09:30 +08:00
2d372e35e2
[Distributed] Kill training jobs in distributed training ( #2881 )
...
* kill training jobs.
* update.
* fix.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-73-81.ec2.internal >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-05-01 12:21:29 +08:00
bfbbefa793
[Distributed] Fix a bug in multiprocessing sampling. ( #2826 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-73-81.ec2.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-04-08 01:29:33 -07:00
86229d426f
[Distributed] Automatically setting the number of OMP threads for trainers ( #2812 )
...
* set omp thread.
* add comment.
* fix.
2021-04-04 11:50:28 -07:00
e36c5db614
[Distributed] Simplify distributed API ( #2775 )
...
* remove num_workers.
* remove num_workers.
* remove num_workers.
* remove num-servers.
* update error message.
* update docstring.
* fix docs.
* fix tests.
* fix test.
* fix.
* print messages in test.
* fix.
* fix test.
* fix.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal >
2021-03-29 19:19:23 -07:00
4327d7125d
[Distributed] Check num_workers and num_samplers ( #2108 )
...
* check num_workers
* update
* update
* update
* update
* update
* update
2020-08-27 12:42:25 +08:00
f5d8fa8476
[Distributed] Fix all arguments to the format of xx_xxx ( #2005 )
...
* update
* update
2020-08-12 14:45:30 +08:00
4efa332044
[Distributed] Move server start code to initialize. ( #2002 )
...
* move server start code to initialize.
* fix.
* fix lint.
* fix examples.
* add more checks.
2020-08-11 16:58:18 -07:00
d340ea3aa8
[Distributed] Remove server_count from ip_config.txt ( #1985 )
...
* remove server_count from ip_config.txt
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update dist_context.py
* fix lint.
* make it work for multiple spaces.
* update ip_config.txt.
* fix examples.
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* udpate
* update
* update
* update
* update
* update
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal >
2020-08-11 17:41:48 +08:00
ee30b2aadd
[Distributed] Fix standalone ( #1974 )
...
* fix tests.
* fix.
* remove a test.
* make code work in the standalone mode.
* fix example.
* more fix.
* make DistDataloader work with num_workers=0
* fix DistDataloader tests.
* fix.
* fix lint.
* fix cleanup.
* fix test
* remove unnecessary code.
* remove tests.
* fix.
* fix.
* fix.
* fix example
* fix.
* fix.
* fix launch script.
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal >
2020-08-10 15:49:28 -07:00
b9ef70e5ac
[Distributed] Set the number of threads correctly to speed up ( #1976 )
...
* temp fix omp.
* set server threads.
* add CAPI to set up OMP threads.
* fix.
* fix.
* update namesapce.
* set cpi properly.
* allow to config num worker threads.
* set #threads.
* fix.
2020-08-08 19:30:53 -07:00
f0fbbc1632
[Distributed] Fix the launch script. ( #1977 )
...
* update launch script
* check the correctness of launch script.
* fix.
2020-08-08 09:56:25 -07:00
4be4b13424
[Distributed] add copy_partitions.py ( #1866 )
...
* fix bugs.
* eval on both vaidation and testing.
* add script.
* update.
* update launch.
* make train_dist.py independent.
* update readme.
* update readme.
* update readme.
* update readme.
* generate undirected graph.
* rename conf_file to part_config
* use rsync
* make train_dist independent.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-115.us-west-2.compute.internal >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-07-31 11:11:56 -07:00
bcb988bdea
[Distributed] Small fix on launch script ( #1867 )
...
* update
* update
* update
* update
2020-07-27 13:44:14 +08:00
7645c6609a
fix launch. ( #1822 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal >
2020-07-17 16:25:26 -07:00
ca9d3216d0
[Distributed] Distributed launching script ( #1772 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix launch script.
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-07-16 10:20:57 -07:00
Da Zheng