payload: {"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/26225","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/26225/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/26225/comments","events_url":"https://api.github.com/repos/apache/spark/issues/26225/events","html_url":"https://github.com/apache/spark/pull/26225","id":511207885,"node_id":"MDExOlB1bGxSZXF1ZXN0MzMxNDQ3NTA0","number":26225,"title":"[SPARK-29568][SS] Stop existing running streams when a new stream is launched","user":{"login":"brkyvz","id":5243515,"node_id":"MDQ6VXNlcjUyNDM1MTU=","avatar_url":"https://avatars1.githubusercontent.com/u/5243515?v=4","gravatar_id":"","url":"https://api.github.com/users/brkyvz","html_url":"https://github.com/brkyvz","followers_url":"https://api.github.com/users/brkyvz/followers","following_url":"https://api.github.com/users/brkyvz/following{/other_user}","gists_url":"https://api.github.com/users/brkyvz/gists{/gist_id}","starred_url":"https://api.github.com/users/brkyvz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brkyvz/subscriptions","organizations_url":"https://api.github.com/users/brkyvz/orgs","repos_url":"https://api.github.com/users/brkyvz/repos","events_url":"https://api.github.com/users/brkyvz/events{/privacy}","received_events_url":"https://api.github.com/users/brkyvz/received_events","type":"User","site_admin":false},"labels":[{"id":1406587328,"node_id":"MDU6TGFiZWwxNDA2NTg3MzI4","url":"https://api.github.com/repos/apache/spark/labels/STRUCTURED%20STREAMING","name":"STRUCTURED STREAMING","color":"ededed","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":19,"created_at":"2019-10-23T09:50:14Z","updated_at":"2019-11-07T17:02:05Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/26225","html_url":"https://github.com/apache/spark/pull/26225","diff_url":"https://github.com/apache/spark/pull/26225.diff","patch_url":"https://github.com/apache/spark/pull/26225.patch"},"body":"### What changes were proposed in this pull request?\r\n\r\nThis PR adds a SQL Conf: `spark.sql.streaming.stopExistingDuplicateStream`. When this conf is `true` (by default it is), an already running stream will be stopped, if a new copy gets launched on the same checkpoint location.\r\n\r\n### Why are the changes needed?\r\n\r\nIn multi-tenant environments where you have multiple SparkSessions, you can accidentally start multiple copies of the same stream (i.e. streams using the same checkpoint location). This will cause all new instantiations of the new stream to fail. However, sometimes you may want to turn off the old stream, as the old stream may have turned into a zombie (you no longer have access to the query handle or SparkSession).\r\n\r\nIt would be nice to have a SQL flag that allows the stopping of the old stream for such zombie cases.\r\n\r\n### Does this PR introduce any user-facing change?\r\n\r\nYes. Now by default, if you launch a new copy of an already running stream on a multi-tenant cluster, the existing stream will be stopped.\r\n\r\n### How was this patch tested?\r\n\r\nUnit tests in StreamingQueryManagerSuite"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/551170641","html_url":"https://github.com/apache/spark/pull/26225#issuecomment-551170641","issue_url":"https://api.github.com/repos/apache/spark/issues/26225","id":551170641,"node_id":"MDEyOklzc3VlQ29tbWVudDU1MTE3MDY0MQ==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2019-11-07T17:02:04Z","updated_at":"2019-11-07T17:02:04Z","author_association":"NONE","body":"Merged build finished. Test PASSed."}}
other: {"actor":{"display_login":"AmplabJenkins"}}