payload: {"action":"created","issue":{"url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/issues/196","repository_url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark","labels_url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/issues/196/labels{/name}","comments_url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/issues/196/comments","events_url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/issues/196/events","html_url":"https://github.com/facebookresearch/maskrcnn-benchmark/pull/196","id":383221095,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMyNzQzNzMw","number":196,"title":"Initial mixed-precision training","user":{"login":"slayton58","id":4992598,"node_id":"MDQ6VXNlcjQ5OTI1OTg=","avatar_url":"https://avatars3.githubusercontent.com/u/4992598?v=4","gravatar_id":"","url":"https://api.github.com/users/slayton58","html_url":"https://github.com/slayton58","followers_url":"https://api.github.com/users/slayton58/followers","following_url":"https://api.github.com/users/slayton58/following{/other_user}","gists_url":"https://api.github.com/users/slayton58/gists{/gist_id}","starred_url":"https://api.github.com/users/slayton58/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/slayton58/subscriptions","organizations_url":"https://api.github.com/users/slayton58/orgs","repos_url":"https://api.github.com/users/slayton58/repos","events_url":"https://api.github.com/users/slayton58/events{/privacy}","received_events_url":"https://api.github.com/users/slayton58/received_events","type":"User","site_admin":false},"labels":[{"id":1103191908,"node_id":"MDU6TGFiZWwxMTAzMTkxOTA4","url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/labels/CLA%20Signed","name":"CLA Signed","color":"009900","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":22,"created_at":"2018-11-21T17:21:05Z","updated_at":"2019-04-17T15:07:36Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/pulls/196","html_url":"https://github.com/facebookresearch/maskrcnn-benchmark/pull/196","diff_url":"https://github.com/facebookresearch/maskrcnn-benchmark/pull/196.diff","patch_url":"https://github.com/facebookresearch/maskrcnn-benchmark/pull/196.patch"},"body":"This PR adds initial mixed-precision training support via [apex.amp](https://github.com/NVIDIA/apex/tree/master/apex/amp).\r\n\r\nMixed-precision is controlled with the `SOLVER.MIXED_PRECISION` config argument.\r\n\r\nAlong with `apex.amp` support, I've moved `DistributedDataParallel` to `apex.DistributedDataParallel` as this is what we've been using to good effect over the last few months.\r\n\r\nPlease note that this does add the `apex` package as a requirement."},"comment":{"url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/issues/comments/484130811","html_url":"https://github.com/facebookresearch/maskrcnn-benchmark/pull/196#issuecomment-484130811","issue_url":"https://api.github.com/repos/facebookresearch/maskrcnn-benchmark/issues/196","id":484130811,"node_id":"MDEyOklzc3VlQ29tbWVudDQ4NDEzMDgxMQ==","user":{"login":"LeviViana","id":17881241,"node_id":"MDQ6VXNlcjE3ODgxMjQx","avatar_url":"https://avatars1.githubusercontent.com/u/17881241?v=4","gravatar_id":"","url":"https://api.github.com/users/LeviViana","html_url":"https://github.com/LeviViana","followers_url":"https://api.github.com/users/LeviViana/followers","following_url":"https://api.github.com/users/LeviViana/following{/other_user}","gists_url":"https://api.github.com/users/LeviViana/gists{/gist_id}","starred_url":"https://api.github.com/users/LeviViana/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/LeviViana/subscriptions","organizations_url":"https://api.github.com/users/LeviViana/orgs","repos_url":"https://api.github.com/users/LeviViana/repos","events_url":"https://api.github.com/users/LeviViana/events{/privacy}","received_events_url":"https://api.github.com/users/LeviViana/received_events","type":"User","site_admin":false},"created_at":"2019-04-17T15:07:36Z","updated_at":"2019-04-17T15:07:36Z","author_association":"COLLABORATOR","body":"I've made some tests on 4 x 2080Ti. I didn't experience any improvement in training time. In my tests, I set `_C.DYTPE = \"float16\"` vs `_C.DYTPE = \"float32\"`. I tested :\r\n\r\n- `e2e_faster_rcnn_R_50_FPN_1x.yaml` for 500 iterations\r\n- `e2e_mask_rcnn_X_101_32x8d_FPN_1x.yaml` for 500 iterations\r\n\r\n@slayton58 what could be going wrong on these tests ? Could you provide some benchmarks on the improvements you've achieved ?"}}
other: {"actor":{"display_login":"LeviViana"}}