payload: {"action":"created","issue":{"url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/issues/14","repository_url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation","labels_url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/issues/14/labels{/name}","comments_url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/issues/14/comments","events_url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/issues/14/events","html_url":"https://github.com/ToniRV/spark_vio_evaluation/pull/14","id":519058552,"node_id":"MDExOlB1bGxSZXF1ZXN0MzM3ODY5MTU1","number":14,"title":"Even bigger refactor - use DatasetRunner class","user":{"login":"marcusabate","id":32229998,"node_id":"MDQ6VXNlcjMyMjI5OTk4","avatar_url":"https://avatars3.githubusercontent.com/u/32229998?v=4","gravatar_id":"","url":"https://api.github.com/users/marcusabate","html_url":"https://github.com/marcusabate","followers_url":"https://api.github.com/users/marcusabate/followers","following_url":"https://api.github.com/users/marcusabate/following{/other_user}","gists_url":"https://api.github.com/users/marcusabate/gists{/gist_id}","starred_url":"https://api.github.com/users/marcusabate/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marcusabate/subscriptions","organizations_url":"https://api.github.com/users/marcusabate/orgs","repos_url":"https://api.github.com/users/marcusabate/repos","events_url":"https://api.github.com/users/marcusabate/events{/privacy}","received_events_url":"https://api.github.com/users/marcusabate/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"marcusabate","id":32229998,"node_id":"MDQ6VXNlcjMyMjI5OTk4","avatar_url":"https://avatars3.githubusercontent.com/u/32229998?v=4","gravatar_id":"","url":"https://api.github.com/users/marcusabate","html_url":"https://github.com/marcusabate","followers_url":"https://api.github.com/users/marcusabate/followers","following_url":"https://api.github.com/users/marcusabate/following{/other_user}","gists_url":"https://api.github.com/users/marcusabate/gists{/gist_id}","starred_url":"https://api.github.com/users/marcusabate/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marcusabate/subscriptions","organizations_url":"https://api.github.com/users/marcusabate/orgs","repos_url":"https://api.github.com/users/marcusabate/repos","events_url":"https://api.github.com/users/marcusabate/events{/privacy}","received_events_url":"https://api.github.com/users/marcusabate/received_events","type":"User","site_admin":false},"assignees":[{"login":"marcusabate","id":32229998,"node_id":"MDQ6VXNlcjMyMjI5OTk4","avatar_url":"https://avatars3.githubusercontent.com/u/32229998?v=4","gravatar_id":"","url":"https://api.github.com/users/marcusabate","html_url":"https://github.com/marcusabate","followers_url":"https://api.github.com/users/marcusabate/followers","following_url":"https://api.github.com/users/marcusabate/following{/other_user}","gists_url":"https://api.github.com/users/marcusabate/gists{/gist_id}","starred_url":"https://api.github.com/users/marcusabate/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marcusabate/subscriptions","organizations_url":"https://api.github.com/users/marcusabate/orgs","repos_url":"https://api.github.com/users/marcusabate/repos","events_url":"https://api.github.com/users/marcusabate/events{/privacy}","received_events_url":"https://api.github.com/users/marcusabate/received_events","type":"User","site_admin":false}],"milestone":null,"comments":17,"created_at":"2019-11-07T05:57:51Z","updated_at":"2019-11-24T04:24:14Z","closed_at":null,"author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/pulls/14","html_url":"https://github.com/ToniRV/spark_vio_evaluation/pull/14","diff_url":"https://github.com/ToniRV/spark_vio_evaluation/pull/14.diff","patch_url":"https://github.com/ToniRV/spark_vio_evaluation/pull/14.patch"},"body":"This PR adds some major refactoring:\r\n\r\n- Separate `DatasetRunner` from `DatasetEvaluator`\r\n- Add several helper functions for both classes\r\n- Collapse shared functionality between pgo evaluation and vio-only evaluation\r\n- Add `use_lcd` and `plot_vio_and_pgo` parameters on a per-dataset basis\r\n- `results.yaml` is now `results_vio.yaml`, and there is an equivalent `results_pgo.yaml` file for cases in which `use_lcd` is set to true.\r\n\r\nThe `use_lcd` parameter is sent to the executable when the dataset is run and activates the `use_lcd` flag.\r\nThe `plot_vio_and_pgo` parameter tells the `DatasetEvaluator` to add plots for both the PGO and VIO-only performance to `plots.pdf` and `plots.eps`.\r\n\r\nToDo:\r\n- [x] Currently, because `DataEvaluator` moves output files from the temporary location to the permanent one before doing analysis, you cannot use commandline argument `-a` without also using `-r`. You must run the dataset to log output to the temporary location so that it can then be moved properly. If you wish to run evaluation only, ensure that the output logs are located at the temporary location.\r\n- [ ] The results yaml files store numpy types that are not human readable. Other loaders would struggle to parse the info. We need a \"round-trip\" yaml dump that converts all data to python primitives.\r\n- [x] `confirm_overwrite` is never actually used as a parameter. We should get rid of the option."},"comment":{"url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/issues/comments/557856562","html_url":"https://github.com/ToniRV/spark_vio_evaluation/pull/14#issuecomment-557856562","issue_url":"https://api.github.com/repos/ToniRV/spark_vio_evaluation/issues/14","id":557856562,"node_id":"MDEyOklzc3VlQ29tbWVudDU1Nzg1NjU2Mg==","user":{"login":"ToniRV","id":9856163,"node_id":"MDQ6VXNlcjk4NTYxNjM=","avatar_url":"https://avatars3.githubusercontent.com/u/9856163?v=4","gravatar_id":"","url":"https://api.github.com/users/ToniRV","html_url":"https://github.com/ToniRV","followers_url":"https://api.github.com/users/ToniRV/followers","following_url":"https://api.github.com/users/ToniRV/following{/other_user}","gists_url":"https://api.github.com/users/ToniRV/gists{/gist_id}","starred_url":"https://api.github.com/users/ToniRV/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ToniRV/subscriptions","organizations_url":"https://api.github.com/users/ToniRV/orgs","repos_url":"https://api.github.com/users/ToniRV/repos","events_url":"https://api.github.com/users/ToniRV/events{/privacy}","received_events_url":"https://api.github.com/users/ToniRV/received_events","type":"User","site_admin":false},"created_at":"2019-11-24T04:24:14Z","updated_at":"2019-11-24T04:24:14Z","author_association":"OWNER","body":"Feel free to merge when ready."}}
other: {"actor":{"display_login":"ToniRV"}}