payload: {"action":"created","issue":{"url":"https://api.github.com/repos/greenplum-db/gpdb/issues/9223","repository_url":"https://api.github.com/repos/greenplum-db/gpdb","labels_url":"https://api.github.com/repos/greenplum-db/gpdb/issues/9223/labels{/name}","comments_url":"https://api.github.com/repos/greenplum-db/gpdb/issues/9223/comments","events_url":"https://api.github.com/repos/greenplum-db/gpdb/issues/9223/events","html_url":"https://github.com/greenplum-db/gpdb/pull/9223","id":536223276,"node_id":"MDExOlB1bGxSZXF1ZXN0MzUxNzkyMTM0","number":9223,"title":"Multiple Distinct-qualified Aggregation(Multi-DQA) MPP execution method ","user":{"login":"weinan003","id":11862896,"node_id":"MDQ6VXNlcjExODYyODk2","avatar_url":"https://avatars3.githubusercontent.com/u/11862896?v=4","gravatar_id":"","url":"https://api.github.com/users/weinan003","html_url":"https://github.com/weinan003","followers_url":"https://api.github.com/users/weinan003/followers","following_url":"https://api.github.com/users/weinan003/following{/other_user}","gists_url":"https://api.github.com/users/weinan003/gists{/gist_id}","starred_url":"https://api.github.com/users/weinan003/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/weinan003/subscriptions","organizations_url":"https://api.github.com/users/weinan003/orgs","repos_url":"https://api.github.com/users/weinan003/repos","events_url":"https://api.github.com/users/weinan003/events{/privacy}","received_events_url":"https://api.github.com/users/weinan003/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2019-12-11T08:45:18Z","updated_at":"2019-12-11T09:46:35Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/greenplum-db/gpdb/pulls/9223","html_url":"https://github.com/greenplum-db/gpdb/pull/9223","diff_url":"https://github.com/greenplum-db/gpdb/pull/9223.diff","patch_url":"https://github.com/greenplum-db/gpdb/pull/9223.patch"},"body":"This PR bring Multi-DQA MPP execution to master again after 9.6 merged but a new strategy introduced. A new aggregate, `AGG_TUP_SPLIT`, divides each input tuple to `n` output tuples (`n` is the number of DQA exprs).  Each output tuple only contains one DQA expr and all GROUP BY expr, so that we can remove multiple DQA in one path. After previous work, a normal two-stage aggregate without distinct attaches on top. \r\n\r\nFor this method, it is a limitation that GROUP BY clause can not be the same as DQA expr. The two-stage aggregate will get the wrong result since  GROUP BY keeps its expr in every tuple. To deal with this corner case, I declare a new expr named ShadowExpr. It just simply copy its major expr's value and cheat executor.\r\n```\r\nSELECT count(distinct a), count(distinct b) FROM foo GROUP BY a;\r\n```\r\nthis query will translate as\r\n```\r\nSELECT count(distinct a), count(distinct b) FROM foo GROUP BY a';\r\n(a' is the Shadow Expr for a, and all (a')'s value == a value)\r\n```\r\n\r\n \r\nCo-authored-by: Adam Li <ali@pivotal.io>\r\nInspired by: Heikki Linnakangas <heikki.linnakangas@iki.fi>\r\n## Here are some reminders before you submit the pull request\r\n- [x] Add tests for the change\r\n- [ ] Document changes\r\n- [ ] Communicate in the mailing list if needed\r\n- [x] Pass `make installcheck`\r\n- [ ] Review a PR in return to support the community\r\n"},"comment":{"url":"https://api.github.com/repos/greenplum-db/gpdb/issues/comments/564462150","html_url":"https://github.com/greenplum-db/gpdb/pull/9223#issuecomment-564462150","issue_url":"https://api.github.com/repos/greenplum-db/gpdb/issues/9223","id":564462150,"node_id":"MDEyOklzc3VlQ29tbWVudDU2NDQ2MjE1MA==","user":{"login":"hlinnaka","id":191602,"node_id":"MDQ6VXNlcjE5MTYwMg==","avatar_url":"https://avatars2.githubusercontent.com/u/191602?v=4","gravatar_id":"","url":"https://api.github.com/users/hlinnaka","html_url":"https://github.com/hlinnaka","followers_url":"https://api.github.com/users/hlinnaka/followers","following_url":"https://api.github.com/users/hlinnaka/following{/other_user}","gists_url":"https://api.github.com/users/hlinnaka/gists{/gist_id}","starred_url":"https://api.github.com/users/hlinnaka/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hlinnaka/subscriptions","organizations_url":"https://api.github.com/users/hlinnaka/orgs","repos_url":"https://api.github.com/users/hlinnaka/repos","events_url":"https://api.github.com/users/hlinnaka/events{/privacy}","received_events_url":"https://api.github.com/users/hlinnaka/received_events","type":"User","site_admin":false},"created_at":"2019-12-11T09:46:35Z","updated_at":"2019-12-11T09:46:35Z","author_association":"MEMBER","body":"The cost estimate for the TupleSplit node is not good:\r\n```\r\npostgres=#  explain analyze select count(distinct d), count(distinct dt) from dqa_t1;\r\n                                                                     QUERY PLAN                                                                      \r\n-----------------------------------------------------------------------------------------------------------------------------------------------------\r\n Finalize Aggregate  (cost=4.20..4.21 rows=1 width=16) (actual time=2.126..2.126 rows=1 loops=1)\r\n   ->  Gather Motion 3:1  (slice2; segments: 3)  (cost=4.16..4.19 rows=1 width=16) (actual time=2.105..2.114 rows=3 loops=1)\r\n         ->  Partial Aggregate  (cost=4.16..4.17 rows=1 width=16) (actual time=0.753..0.753 rows=1 loops=1)\r\n               ->  Streaming HashAggregate  (cost=4.12..4.15 rows=1 width=8) (actual time=0.755..0.802 rows=24 loops=1)\r\n                     Group Key: d, dt\r\n                     Extra Text: (seg1)   Hash chain length 1.3 avg, 4 max, using 18 of 32 buckets; total 0 expansions.\r\n                     ->  Redistribute Motion 3:3  (slice1; segments: 3)  (cost=4.00..4.09 rows=1 width=8) (actual time=0.569..0.626 rows=86 loops=1)\r\n                           Hash Key: d, dt\r\n                           ->  TupleSplit  (cost=4.00..4.03 rows=1 width=8) (actual time=0.026..0.060 rows=80 loops=1)\r\n                                 Split by Col: d, dt\r\n                                 ->  Seq Scan on dqa_t1  (cost=0.00..4.00 rows=34 width=8) (actual time=0.023..0.036 rows=40 loops=1)\r\n Planning time: 1.232 ms\r\n   (slice0)    Executor memory: 144K bytes.\r\n   (slice1)    Executor memory: 59K bytes avg x 3 workers, 59K bytes max (seg0).\r\n   (slice2)    Executor memory: 162K bytes avg x 3 workers, 168K bytes max (seg1).\r\n Memory used:  128000kB\r\n Optimizer: Postgres query optimizer\r\n Execution time: 16.441 ms\r\n(18 rows)\r\n```\r\nThe \"rows\" estimate of the TupleSplit node is 1, which is surely not right. I think the correct estimate would be \"(# of rows from input) * (# of DQAs)\"."}}
other: {"actor":{"display_login":"hlinnaka"}}