payload: {"action":"created","comment":{"url":"https://api.github.com/repos/apache/flink/pulls/comments/334365556","pull_request_review_id":301154483,"id":334365556,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDMzNDM2NTU1Ng==","diff_hunk":"@@ -246,23 +256,23 @@ public void allocateSlotsAndDeploy(final Collection<ExecutionVertexDeploymentOpt\n \t\t}\n \t}\n \n+\tprivate void validateDeploymentOptions(final Collection<ExecutionVertexDeploymentOption> deploymentOptions) {\n+\t\tdeploymentOptions.stream()\n+\t\t\t.map(ExecutionVertexDeploymentOption::getExecutionVertexId)\n+\t\t\t.map(this::getExecutionVertex)\n+\t\t\t.map(ExecutionVertex::getExecutionState)\n+\t\t\t.forEach(state -> checkState(\n+\t\t\t\tstate == ExecutionState.CREATED,\n+\t\t\t\t\"vertex to schedule must be in CREATED state\"));","path":"flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultScheduler.java","position":null,"original_position":53,"commit_id":"530c6772299b02d9cf1333f6687b1f8df11e4278","original_commit_id":"c602f6a3deb290f5f6106deefee0a02645126034","user":{"login":"zhuzhurk","id":5869249,"node_id":"MDQ6VXNlcjU4NjkyNDk=","avatar_url":"https://avatars2.githubusercontent.com/u/5869249?v=4","gravatar_id":"","url":"https://api.github.com/users/zhuzhurk","html_url":"https://github.com/zhuzhurk","followers_url":"https://api.github.com/users/zhuzhurk/followers","following_url":"https://api.github.com/users/zhuzhurk/following{/other_user}","gists_url":"https://api.github.com/users/zhuzhurk/gists{/gist_id}","starred_url":"https://api.github.com/users/zhuzhurk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhuzhurk/subscriptions","organizations_url":"https://api.github.com/users/zhuzhurk/orgs","repos_url":"https://api.github.com/users/zhuzhurk/repos","events_url":"https://api.github.com/users/zhuzhurk/events{/privacy}","received_events_url":"https://api.github.com/users/zhuzhurk/received_events","type":"User","site_admin":false},"body":"Ok. Done.","created_at":"2019-10-14T08:13:44Z","updated_at":"2019-10-14T08:13:44Z","html_url":"https://github.com/apache/flink/pull/9860#discussion_r334365556","pull_request_url":"https://api.github.com/repos/apache/flink/pulls/9860","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/comments/334365556"},"html":{"href":"https://github.com/apache/flink/pull/9860#discussion_r334365556"},"pull_request":{"href":"https://api.github.com/repos/apache/flink/pulls/9860"}},"in_reply_to_id":334354868},"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/9860","id":326120110,"node_id":"MDExOlB1bGxSZXF1ZXN0MzI2MTIwMTEw","html_url":"https://github.com/apache/flink/pull/9860","diff_url":"https://github.com/apache/flink/pull/9860.diff","patch_url":"https://github.com/apache/flink/pull/9860.patch","issue_url":"https://api.github.com/repos/apache/flink/issues/9860","number":9860,"state":"open","locked":false,"title":"[FLINK-14331][runtime] Reset vertices right after they transition to terminated states","user":{"login":"zhuzhurk","id":5869249,"node_id":"MDQ6VXNlcjU4NjkyNDk=","avatar_url":"https://avatars2.githubusercontent.com/u/5869249?v=4","gravatar_id":"","url":"https://api.github.com/users/zhuzhurk","html_url":"https://github.com/zhuzhurk","followers_url":"https://api.github.com/users/zhuzhurk/followers","following_url":"https://api.github.com/users/zhuzhurk/following{/other_user}","gists_url":"https://api.github.com/users/zhuzhurk/gists{/gist_id}","starred_url":"https://api.github.com/users/zhuzhurk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhuzhurk/subscriptions","organizations_url":"https://api.github.com/users/zhuzhurk/orgs","repos_url":"https://api.github.com/users/zhuzhurk/repos","events_url":"https://api.github.com/users/zhuzhurk/events{/privacy}","received_events_url":"https://api.github.com/users/zhuzhurk/received_events","type":"User","site_admin":false},"body":"\r\n## What is the purpose of the change\r\n\r\nCurrently in DefaultScheduler, tasks to restart will remain in terminated state until they are re-scheduled by the SchedulingStrategy.\r\nThis behavior may cause 2 problems:\r\n1. Failed/Canceled tasks are possibly not be able to be restarted in lazy scheduling. e.g. The job A1-pipelined->B1 fails. And only A1 will be re-scheduled on restartTasks() since the inputs of B1 are not ready. B1 should be scheduled later on the partition consumable event from restarted A1. But the terminal state of B1 will prevent B1 from being scheduled.\r\n2. Keeping a task in FAILED/CANCELED state for a long time can happen if it takes a long time for its inputs to become ready again. This is also not friendly to users, which may cause confusions.\r\n\r\nThis PR to reset vertices right after they transition to terminated states.\r\n\r\n\r\n## Brief change log\r\n\r\n  - *Change DefaultScheduler to reset vertex right after the vertex is terminated*\r\n  - *Change DefaultScheduler to schedule vertices in CREATED state only*\r\n  - *hotfix changes see each commits*\r\n\r\n\r\n## Verifying this change\r\n\r\n\r\nThis change is already covered by existing tests.\r\n\r\nThis change also added some tests:\r\n  - *Added UT to verify the changes on DefaultScheduler*\r\n\r\n## Does this pull request potentially affect one of the following parts:\r\n\r\n  - Dependencies (does it add or upgrade a dependency): (yes / **no**)\r\n  - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)\r\n  - The serializers: (yes / **no** / don't know)\r\n  - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)\r\n  - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (**yes** / no / don't know)\r\n  - The S3 file system connector: (yes / **no** / don't know)\r\n\r\n## Documentation\r\n\r\n  - Does this pull request introduce a new feature? (yes / **no**)\r\n  - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)\r\n","created_at":"2019-10-09T08:02:45Z","updated_at":"2019-10-14T08:13:44Z","closed_at":null,"merged_at":null,"merge_commit_sha":"53afaa1f4aa942f7db10eda7524909dbb5c1ffd1","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":1272994322,"node_id":"MDU6TGFiZWwxMjcyOTk0MzIy","url":"https://api.github.com/repos/apache/flink/labels/component=Runtime/Coordination","name":"component=Runtime/Coordination","color":"175fb7","default":false},{"id":1244454690,"node_id":"MDU6TGFiZWwxMjQ0NDU0Njkw","url":"https://api.github.com/repos/apache/flink/labels/review=description?","name":"review=description?","color":"bcf5db","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/apache/flink/pulls/9860/commits","review_comments_url":"https://api.github.com/repos/apache/flink/pulls/9860/comments","review_comment_url":"https://api.github.com/repos/apache/flink/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/flink/issues/9860/comments","statuses_url":"https://api.github.com/repos/apache/flink/statuses/530c6772299b02d9cf1333f6687b1f8df11e4278","head":{"label":"zhuzhurk:FLINK_14331_reset_vertices_eagerly","ref":"FLINK_14331_reset_vertices_eagerly","sha":"530c6772299b02d9cf1333f6687b1f8df11e4278","user":{"login":"zhuzhurk","id":5869249,"node_id":"MDQ6VXNlcjU4NjkyNDk=","avatar_url":"https://avatars2.githubusercontent.com/u/5869249?v=4","gravatar_id":"","url":"https://api.github.com/users/zhuzhurk","html_url":"https://github.com/zhuzhurk","followers_url":"https://api.github.com/users/zhuzhurk/followers","following_url":"https://api.github.com/users/zhuzhurk/following{/other_user}","gists_url":"https://api.github.com/users/zhuzhurk/gists{/gist_id}","starred_url":"https://api.github.com/users/zhuzhurk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhuzhurk/subscriptions","organizations_url":"https://api.github.com/users/zhuzhurk/orgs","repos_url":"https://api.github.com/users/zhuzhurk/repos","events_url":"https://api.github.com/users/zhuzhurk/events{/privacy}","received_events_url":"https://api.github.com/users/zhuzhurk/received_events","type":"User","site_admin":false},"repo":{"id":160664766,"node_id":"MDEwOlJlcG9zaXRvcnkxNjA2NjQ3NjY=","name":"flink","full_name":"zhuzhurk/flink","private":false,"owner":{"login":"zhuzhurk","id":5869249,"node_id":"MDQ6VXNlcjU4NjkyNDk=","avatar_url":"https://avatars2.githubusercontent.com/u/5869249?v=4","gravatar_id":"","url":"https://api.github.com/users/zhuzhurk","html_url":"https://github.com/zhuzhurk","followers_url":"https://api.github.com/users/zhuzhurk/followers","following_url":"https://api.github.com/users/zhuzhurk/following{/other_user}","gists_url":"https://api.github.com/users/zhuzhurk/gists{/gist_id}","starred_url":"https://api.github.com/users/zhuzhurk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhuzhurk/subscriptions","organizations_url":"https://api.github.com/users/zhuzhurk/orgs","repos_url":"https://api.github.com/users/zhuzhurk/repos","events_url":"https://api.github.com/users/zhuzhurk/events{/privacy}","received_events_url":"https://api.github.com/users/zhuzhurk/received_events","type":"User","site_admin":false},"html_url":"https://github.com/zhuzhurk/flink","description":"Apache Flink","fork":true,"url":"https://api.github.com/repos/zhuzhurk/flink","forks_url":"https://api.github.com/repos/zhuzhurk/flink/forks","keys_url":"https://api.github.com/repos/zhuzhurk/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zhuzhurk/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zhuzhurk/flink/teams","hooks_url":"https://api.github.com/repos/zhuzhurk/flink/hooks","issue_events_url":"https://api.github.com/repos/zhuzhurk/flink/issues/events{/number}","events_url":"https://api.github.com/repos/zhuzhurk/flink/events","assignees_url":"https://api.github.com/repos/zhuzhurk/flink/assignees{/user}","branches_url":"https://api.github.com/repos/zhuzhurk/flink/branches{/branch}","tags_url":"https://api.github.com/repos/zhuzhurk/flink/tags","blobs_url":"https://api.github.com/repos/zhuzhurk/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zhuzhurk/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zhuzhurk/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/zhuzhurk/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zhuzhurk/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/zhuzhurk/flink/languages","stargazers_url":"https://api.github.com/repos/zhuzhurk/flink/stargazers","contributors_url":"https://api.github.com/repos/zhuzhurk/flink/contributors","subscribers_url":"https://api.github.com/repos/zhuzhurk/flink/subscribers","subscription_url":"https://api.github.com/repos/zhuzhurk/flink/subscription","commits_url":"https://api.github.com/repos/zhuzhurk/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/zhuzhurk/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/zhuzhurk/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/zhuzhurk/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/zhuzhurk/flink/contents/{+path}","compare_url":"https://api.github.com/repos/zhuzhurk/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zhuzhurk/flink/merges","archive_url":"https://api.github.com/repos/zhuzhurk/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zhuzhurk/flink/downloads","issues_url":"https://api.github.com/repos/zhuzhurk/flink/issues{/number}","pulls_url":"https://api.github.com/repos/zhuzhurk/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/zhuzhurk/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/zhuzhurk/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zhuzhurk/flink/labels{/name}","releases_url":"https://api.github.com/repos/zhuzhurk/flink/releases{/id}","deployments_url":"https://api.github.com/repos/zhuzhurk/flink/deployments","created_at":"2018-12-06T11:24:20Z","updated_at":"2019-10-14T02:19:46Z","pushed_at":"2019-10-14T08:13:19Z","git_url":"git://github.com/zhuzhurk/flink.git","ssh_url":"git@github.com:zhuzhurk/flink.git","clone_url":"https://github.com/zhuzhurk/flink.git","svn_url":"https://github.com/zhuzhurk/flink","homepage":"","size":207658,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"2f1200623f465b8e35daa86613eee676cac65b2b","user":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":20587599,"node_id":"MDEwOlJlcG9zaXRvcnkyMDU4NzU5OQ==","name":"flink","full_name":"apache/flink","private":false,"owner":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/apache/flink","description":"Apache Flink","fork":false,"url":"https://api.github.com/repos/apache/flink","forks_url":"https://api.github.com/repos/apache/flink/forks","keys_url":"https://api.github.com/repos/apache/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/flink/teams","hooks_url":"https://api.github.com/repos/apache/flink/hooks","issue_events_url":"https://api.github.com/repos/apache/flink/issues/events{/number}","events_url":"https://api.github.com/repos/apache/flink/events","assignees_url":"https://api.github.com/repos/apache/flink/assignees{/user}","branches_url":"https://api.github.com/repos/apache/flink/branches{/branch}","tags_url":"https://api.github.com/repos/apache/flink/tags","blobs_url":"https://api.github.com/repos/apache/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/flink/languages","stargazers_url":"https://api.github.com/repos/apache/flink/stargazers","contributors_url":"https://api.github.com/repos/apache/flink/contributors","subscribers_url":"https://api.github.com/repos/apache/flink/subscribers","subscription_url":"https://api.github.com/repos/apache/flink/subscription","commits_url":"https://api.github.com/repos/apache/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/flink/contents/{+path}","compare_url":"https://api.github.com/repos/apache/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/flink/merges","archive_url":"https://api.github.com/repos/apache/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/flink/downloads","issues_url":"https://api.github.com/repos/apache/flink/issues{/number}","pulls_url":"https://api.github.com/repos/apache/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/flink/labels{/name}","releases_url":"https://api.github.com/repos/apache/flink/releases{/id}","deployments_url":"https://api.github.com/repos/apache/flink/deployments","created_at":"2014-06-07T07:00:10Z","updated_at":"2019-10-14T08:13:31Z","pushed_at":"2019-10-14T08:13:21Z","git_url":"git://github.com/apache/flink.git","ssh_url":"git@github.com:apache/flink.git","clone_url":"https://github.com/apache/flink.git","svn_url":"https://github.com/apache/flink","homepage":"","size":263161,"stargazers_count":10640,"watchers_count":10640,"language":"Java","has_issues":false,"has_projects":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":5698,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":546,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":5698,"open_issues":546,"watchers":10640,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/9860"},"html":{"href":"https://github.com/apache/flink/pull/9860"},"issue":{"href":"https://api.github.com/repos/apache/flink/issues/9860"},"comments":{"href":"https://api.github.com/repos/apache/flink/issues/9860/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/flink/pulls/9860/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/flink/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/flink/pulls/9860/commits"},"statuses":{"href":"https://api.github.com/repos/apache/flink/statuses/530c6772299b02d9cf1333f6687b1f8df11e4278"}},"author_association":"CONTRIBUTOR"}}
other: {"actor":{"display_login":"zhuzhurk"}}