payload: {"action":"created","issue":{"url":"https://api.github.com/repos/mbillingr/arff/issues/5","repository_url":"https://api.github.com/repos/mbillingr/arff","labels_url":"https://api.github.com/repos/mbillingr/arff/issues/5/labels{/name}","comments_url":"https://api.github.com/repos/mbillingr/arff/issues/5/comments","events_url":"https://api.github.com/repos/mbillingr/arff/issues/5/events","html_url":"https://github.com/mbillingr/arff/issues/5","id":472759406,"node_id":"MDU6SXNzdWU0NzI3NTk0MDY=","number":5,"title":"Example of OpenML arff deserialization","user":{"login":"rth","id":630936,"node_id":"MDQ6VXNlcjYzMDkzNg==","avatar_url":"https://avatars0.githubusercontent.com/u/630936?v=4","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"labels":[{"id":940970730,"node_id":"MDU6TGFiZWw5NDA5NzA3MzA=","url":"https://api.github.com/repos/mbillingr/arff/labels/bug","name":"bug","color":"d73a4a","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2019-07-25T09:25:52Z","updated_at":"2019-07-25T11:42:46Z","closed_at":null,"author_association":"NONE","body":"Thanks for this crate!\r\n\r\nI'm trying to load [the MNIST dataset from OpenML](https://www.openml.org/d/554) in Arff format, and so far \r\n```rust\r\nextern crate serde;\r\n\r\nextern crate arff;    \r\nuse std::fs;   \r\nuse serde::Deserialize;\r\n\r\n\r\nfn main() {\r\n    \r\n    let contents = fs::read_to_string(\"/tmp/mnist_784.arff\").expect(\"Error!\");\r\n                                     \r\n    let unnamed_data: Vec<(f32,)> = arff::from_str(&contents).unwrap();   \r\n    println!(\"{:?}\", unnamed_data);\r\n}\r\n```\r\n\r\npanics due to column dtype validation,\r\n```\r\n   Compiling arff-rust-parser v0.1.0 (/home/rth/projects/scikit-learn/arff-rust-parser)\r\n    Finished dev [unoptimized + debuginfo] target(s) in 0.54s\r\n     Running `target/debug/arff-rust-parser`\r\nthread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidColumnType(TextPos { line: 11, column: 18 }, \"\\tREAL\")', src/libcore/result.rs:999:5\r\nnote: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.\r\n```\r\n\r\npossibly because the metadata says that colums are real, while they are integers.\r\n```arff\r\n@ATTRIBUTE pixel1       real\r\n@ATTRIBUTE pixel2       real\r\n@ATTRIBUTE pixel3       real\r\n\r\n@DATA\r\n0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 [..] 0,0,5\r\n```\r\n\r\nIt would be nice if there was an example of loading a dataset from OpenML. A am aware of [openml-rust](https://github.com/mbillingr/openml-rust) but I'm looking for just a fast ARFF parser that I could use as a replacement for [liac-arff](https://github.com/renatopp/liac-arff)."},"comment":{"url":"https://api.github.com/repos/mbillingr/arff/issues/comments/515011120","html_url":"https://github.com/mbillingr/arff/issues/5#issuecomment-515011120","issue_url":"https://api.github.com/repos/mbillingr/arff/issues/5","id":515011120,"node_id":"MDEyOklzc3VlQ29tbWVudDUxNTAxMTEyMA==","user":{"login":"rth","id":630936,"node_id":"MDQ6VXNlcjYzMDkzNg==","avatar_url":"https://avatars0.githubusercontent.com/u/630936?v=4","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"created_at":"2019-07-25T11:42:46Z","updated_at":"2019-07-25T11:42:46Z","author_association":"NONE","body":"Thanks for your response!\r\n\r\nI can confirm that adding a\r\n```rust\r\nlet contents = contents.replace(\"\\t\", \" \");\r\n```\r\nfixes the issue.\r\n \r\nSo on my laptop this loads MNIST in 4.6s as compared to 17s with `liac-arff` which is quite nice (though I guess there is additional overhead to be expected in converting the output to a contiguous array)."}}
other: {"actor":{"display_login":"rth"}}