payload: {"action":"created","issue":{"url":"https://api.github.com/repos/radiocosmology/caput/issues/77","repository_url":"https://api.github.com/repos/radiocosmology/caput","labels_url":"https://api.github.com/repos/radiocosmology/caput/issues/77/labels{/name}","comments_url":"https://api.github.com/repos/radiocosmology/caput/issues/77/comments","events_url":"https://api.github.com/repos/radiocosmology/caput/issues/77/events","html_url":"https://github.com/radiocosmology/caput/pull/77","id":460001245,"node_id":"MDExOlB1bGxSZXF1ZXN0MjkxMjIxMTIw","number":77,"title":"Chunked HDF5 with MPI","user":{"login":"tristpinsm","id":30247013,"node_id":"MDQ6VXNlcjMwMjQ3MDEz","avatar_url":"https://avatars1.githubusercontent.com/u/30247013?v=4","gravatar_id":"","url":"https://api.github.com/users/tristpinsm","html_url":"https://github.com/tristpinsm","followers_url":"https://api.github.com/users/tristpinsm/followers","following_url":"https://api.github.com/users/tristpinsm/following{/other_user}","gists_url":"https://api.github.com/users/tristpinsm/gists{/gist_id}","starred_url":"https://api.github.com/users/tristpinsm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tristpinsm/subscriptions","organizations_url":"https://api.github.com/users/tristpinsm/orgs","repos_url":"https://api.github.com/users/tristpinsm/repos","events_url":"https://api.github.com/users/tristpinsm/events{/privacy}","received_events_url":"https://api.github.com/users/tristpinsm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2019-06-24T17:08:15Z","updated_at":"2019-06-24T19:22:10Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/radiocosmology/caput/pulls/77","html_url":"https://github.com/radiocosmology/caput/pull/77","diff_url":"https://github.com/radiocosmology/caput/pull/77.diff","patch_url":"https://github.com/radiocosmology/caput/pull/77.patch"},"body":"I finished up adding support for chunks/compression to `MPIarray`s last week. I tried to mimick how these options are passed in `h5py`.\r\n\r\nI've tested this in the pipeline with writing out a `SiderealStream` and it seems to work as expected.\r\n\r\nOne thing to note is that collective IO is required to do chunked writes, so I've had to mess around a bit with the logic that figures out when to use this.\r\n\r\nI also have not done any performance testing. This is obviously going to be important, but I know Richard has been investigating this in previous code updates, so I thought I would start the pull request and get some input on how to test it, what to compare it too.\r\n\r\nI'm going to open a separate PR in `draco` with updates to the containers that make use of chunking/compression."},"comment":{"url":"https://api.github.com/repos/radiocosmology/caput/issues/comments/505144875","html_url":"https://github.com/radiocosmology/caput/pull/77#issuecomment-505144875","issue_url":"https://api.github.com/repos/radiocosmology/caput/issues/77","id":505144875,"node_id":"MDEyOklzc3VlQ29tbWVudDUwNTE0NDg3NQ==","user":{"login":"jrs65","id":695206,"node_id":"MDQ6VXNlcjY5NTIwNg==","avatar_url":"https://avatars2.githubusercontent.com/u/695206?v=4","gravatar_id":"","url":"https://api.github.com/users/jrs65","html_url":"https://github.com/jrs65","followers_url":"https://api.github.com/users/jrs65/followers","following_url":"https://api.github.com/users/jrs65/following{/other_user}","gists_url":"https://api.github.com/users/jrs65/gists{/gist_id}","starred_url":"https://api.github.com/users/jrs65/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jrs65/subscriptions","organizations_url":"https://api.github.com/users/jrs65/orgs","repos_url":"https://api.github.com/users/jrs65/repos","events_url":"https://api.github.com/users/jrs65/events{/privacy}","received_events_url":"https://api.github.com/users/jrs65/received_events","type":"User","site_admin":false},"created_at":"2019-06-24T19:22:10Z","updated_at":"2019-06-24T19:22:10Z","author_association":"CONTRIBUTOR","body":"Ooooh awesome. This is mostly great timing! I'm just starting to get worried about the volume of data in the daily processing (600 GB/day) and this would probably be a great saver."}}
other: {"actor":{"display_login":"jrs65"}}