<p>gLiveView can only tell you what your current KES key's start/expiry time is. But another base property of KES key is the number of times your key was rotated. You can check your current key using <code>cardano-cli text-view decode-cbor --in-file &lt;path&gt;/op.cert  | grep int | head -1</code>. Your current cert can only be higher, (not lower) than the one used for your previous blocks.
The upcoming public pgrest API on guild-operators website (currently in alpha stage) will allow you to query your last created block's OCert number for validation too.</p>
<p>PS: You can rotate multiple times to a higher counter (so for example, you can go rotate until counter is 10) just fine, as it will still be higher than your previous block's OCert counter.</p>

CC BY-SA 4.0
2021-06-18T09:08:27.067Z
Thank you. That number shows 2 which is what I would expect given that I rotated KES twice already. Would you mind clarify further because I cannot see how I can use this number for further debugging.
matcheek
CC BY-SA 4.0
2021-06-18T11:09:49.460Z
As per your original thread, it lookslike the counter expected is 6 , not 2. So there is block on chain where your pool's opcert counter was 5 already, and you need to rotate it 4 more times for your index to be higher.
RdLrT
CC BY-SA 4.0
2021-06-18T20:43:18.630Z
I think you can also just open the counter file and manually increase it to something higher.
Danny Tuppeny - CODER
CC BY-SA 4.0
2021-07-02T08:58:56.533Z
My pooled was elected a block leader yesterday. Unfortunately that changes that you proposed and I enacted did not work. Same error, though expected KES evolutions is now 8.
matcheek
CC BY-SA 4.0
2021-07-03T11:09:39.727Z
Please specify both your current counter, new counter in opcert and whether your new KES is generated from same cold key
RdLrT
CC BY-SA 4.0
2021-06-18T07:15:31.217Z
2021-06-18T07:15:31.217Z
RdLrT
<p>We are seeing similar error. We only have rotated the cert once and that was after minting our first block.
@RdLrT this command shows just 1.</p>
<pre><code>cardano-cli.sh &quot;text-view decode-cbor --in-file node.cert&quot;  | grep int | head -1
      01  # int(1)
</code></pre>
<p>But the error says</p>
<p><code>&quot;opCertExpectedKESEvolutions&quot;,String &quot;17&quot;</code>
Does this mean I have to rotate it 16 more times?</p>
<pre><code>cardano-node    | [93f57906:cardano.node.ChainDB:Info:537] [2021-06-17 11:48:08.00 UTC] Valid candidate 1ef53e05e0c8ac9a35519dedc61e974036cb66fbf15a6ecc4c3420fa0db9f8ef at slot 32364102
cardano-node    | [93f57906:cardano.node.Forge:Error:547] [2021-06-17 11:48:08.01 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceForgedInvalidBlock&quot;),(&quot;reason&quot;,Object (fromList [(&quot;error&quot;,Object (fromList [(&quot;error&quot;,Object (fromList [(&quot;failures&quot;,Array [Object (fromList [(&quot;error&quot;,String &quot;Reject&quot;),(&quot;kind&quot;,String &quot;InvalidKesSignatureOCERT&quot;),(&quot;opCertExpectedKESEvolutions&quot;,String &quot;17&quot;),(&quot;opCertKESCurrentPeriod&quot;,String &quot;249&quot;),(&quot;opCertKESStartPeriod&quot;,String &quot;232&quot;)])]),(&quot;kind&quot;,String &quot;ChainTransitionError&quot;)])),(&quot;kind&quot;,String &quot;HeaderProtocolError&quot;)])),(&quot;kind&quot;,String &quot;ValidationError&quot;)])),(&quot;slot&quot;,Number 3.2364197e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
cardano-node    | [93f57906:cardano.node.LeadershipCheck:Info:547] [2021-06-17 11:48:09.00 UTC] {&quot;kind&quot;:&quot;TraceStartLeadershipCheck&quot;,&quot;chainDensity&quot;:4.991215e-2,&quot;slot&quot;:32364198,&quot;delegMapSize&quot;:644407,&quot;utxoSize&quot;:2659418,&quot;credentials&quot;:&quot;Cardano&quot;}
</code></pre>

CC BY-SA 4.0
2021-06-23T17:55:25.813Z
I followed that advice but there is no way to verify if it solves the problem before any blocks are actually minted. I should know the answer within next week or two.
matcheek
CC BY-SA 4.0
2021-06-23T21:28:04.037Z
This appears to be a new question, not really an answer to the original question.  Please consider whether you should make it a separate question, update or delete it.  Welcome aboard, I hope my feedback is helpful.
gRebel
CC BY-SA 4.0
2021-06-24T01:13:32.570Z
@gRebel the problem is exactly the same and didn't want to open a new thread on this to avoid missed context.
user2107938
CC BY-SA 4.0
2021-06-24T06:18:00.443Z
@user2107938 Based on the log message, yes (or you can verify in your dbsync instance too)
RdLrT
CC BY-SA 4.0
2021-06-23T17:19:15.600Z
2021-06-23T17:19:15.600Z
user2107938
<p>If it's not the counter issue mentioned above, I had a similar error in the past:</p>
<blockquote>
<p>[cardano-:cardano.node.Forge:Error:195] [2021-03-15 23:36:47.00 UTC] fromList [(&quot;credentials&quot;,String &quot;Cardano&quot;),(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceForgedInvalidBlock&quot;),(&quot;reason&quot;,Object (fromList [(&quot;kind&quot;,String &quot;ValidationError&quot;),(&quot;error&quot;,Object (fromList [(&quot;kind&quot;,String &quot;HeaderProtocolError&quot;),(&quot;error&quot;,Object (fromList [(&quot;kind&quot;,String &quot;ChainTransitionError&quot;),(&quot;failures&quot;,Array [Object (fromList [(&quot;kind&quot;,String &quot;InvalidKesSignatureOCERT&quot;),(&quot;error&quot;,String &quot;Reject&quot;),(&quot;opCertKESCurrentPeriod&quot;,String &quot;165&quot;),(&quot;opCertExpectedKESEvolutions&quot;,String &quot;1&quot;),(&quot;opCertKESStartPeriod&quot;,String &quot;164&quot;)])])]))]))])),(&quot;slot&quot;,Number 2.1482191e7)]))]</p>
</blockquote>
<p>I believe the cause was that I only copied the updated <code>kes</code> file to the producer, and not the new <code>node.cert</code> file. I re-copied both files over to the producer and restarted, and the blocks started working correctly.</p>

CC BY-SA 4.0
2021-06-19T19:03:12.867Z
Thank you. May I ask you. The steps that I followed only mention about copying node.cert to the prodcution environment. 
https://www.coincashew.com/coins/overview-ada/guide-how-to-build-a-haskell-stakepool-node#18-1-rotate-pools-kes-keys-updating-the-operational-cert-with-a-new-kes-period

What steps did you follow? Did you manage to find any officals Cardano docs for this?
matcheek
CC BY-SA 4.0
2021-06-20T00:46:40.023Z
@matcheek - The issue is already proven to be as per first reply based on log extract and your first reply about current ocert number. It would best for you to rotate your KES 4 more times (so that current is 6 or above) and verify next block assignment is created correctly.
RdLrT
CC BY-SA 4.0
2021-06-20T08:03:07.667Z
@RdLrT Done. Do you know how to verify if KES is correct now before waiting for another minted block?
matcheek
CC BY-SA 4.0
2021-06-20T13:21:10.223Z
I've no idea why it only mentions one, I'm pretty certain you need to copy both! I followed the instructions on the official Cardano site (which has just been updated and is hard to find things now), and I just repeated the bit I assumed was required for KES rotation. I don't know of any way to verify things are correct before minting a block though (I hit this issue on testnet where I had a large delegation and was able to verify fairly quickly).
Danny Tuppeny - CODER
CC BY-SA 4.0
2021-06-18T20:49:08.117Z
2021-06-18T20:49:08.117Z
Danny Tuppeny - CODER
<p>Have recently rotated KES following <a href="https://www.coincashew.com/coins/overview-ada/guide-how-to-build-a-haskell-stakepool-node#18-1-rotate-pools-kes-keys-updating-the-operational-cert-with-a-new-kes-period" rel="nofollow noreferrer">steps</a>.
I cannot get any block to get adopted despite being selected as a slot leader number of times already. As you can see Guild LiveView tool shows that KES kes are valid. But the logs from <code>cardano-node</code> show otherwise.</p>
<p><a href="https://i.sstatic.net/xpriv.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xpriv.png" alt="enter image description here" /></a></p>
<blockquote>
<p>Jun 17 16:39:29 wladyslawa bash[1021152]:</p>
</blockquote>
<blockquote>
<p>[wladysla:cardano.node.ChainDB:Error:82] [2021-06-17 15:39:29.01 UTC]
Invalid block
95def156e23112ebe3f9c6832ac96059d10ff5c446bdbbbc275b2698dfc336d0 at
slot 32378078: ExtValidationErrorHeader (HeaderProtocolError
(HardForkValidationErrFromEra S (S (S (Z (WrapValidationErr
{unwrapValidationErr = ChainTransitionError [OverlayFailure
(OcertFailure (InvalidKesSignatureOCERT 249 243 6 &quot;Reject&quot;))]}))))))</p>
</blockquote>
<h3>More errors:</h3>
<pre><code>maciej@ola:~/Downloads$ grep -ir &quot;:error&quot; logs.txt  -C 1
Jun 17 14:00:38 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 13:00:38.00 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceNodeNotLeader&quot;),(&quot;slot&quot;,Number 3.2368547e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
Jun 17 14:00:38 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:73120] [2021-06-17 13:00:38.48 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Application Exception: 192.168.0.12:6000 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
Jun 17 14:00:38 wladyslawa bash[1021152]: [wladysla:cardano.node.ErrorPolicy:Notice:100] [2021-06-17 13:00:38.48 UTC] IP 192.168.0.12:6000 ErrorPolicySuspendConsumer (Just (ApplicationExceptionTrace ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply))) 20s
--
Jun 17 14:04:25 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 13:04:25.00 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceNodeNotLeader&quot;),(&quot;slot&quot;,Number 3.2368774e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
Jun 17 14:04:25 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:99511] [2021-06-17 13:04:25.20 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Application Exception: 192.168.0.12:6000 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
Jun 17 14:04:25 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Info:99511] [2021-06-17 13:04:25.20 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Closed socket to 192.168.0.12:6000
--
Jun 17 14:05:56 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 13:05:56.00 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceNodeNotLeader&quot;),(&quot;slot&quot;,Number 3.2368865e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
Jun 17 14:05:56 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:99547] [2021-06-17 13:05:56.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Application Exception: 192.168.0.12:6000 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
Jun 17 14:05:56 wladyslawa bash[1021152]: [wladysla:cardano.node.ErrorPolicy:Notice:100] [2021-06-17 13:05:56.21 UTC] IP 192.168.0.12:6000 ErrorPolicySuspendConsumer (Just (ApplicationExceptionTrace ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply))) 20s
--
Jun 17 14:05:57 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Info:102] [2021-06-17 13:05:57.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Skipping peer 192.168.0.12:6000
Jun 17 14:05:57 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:102] [2021-06-17 13:05:57.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Failed to start all required subscriptions
Jun 17 14:05:57 wladyslawa bash[1021152]: [wladysla:cardano.node.Mempool:Info:99577] [2021-06-17 13:05:57.23 UTC] fromList [(&quot;tx&quot;,Object (fromList [(&quot;txid&quot;,String &quot;txid: TxId {_unTxId = SafeHash \&quot;fb5f593592735aa5507b2881705e902611729620968b4c890718cb775ce9d4fc\&quot;}&quot;)])),(&quot;mempoolSize&quot;,Object (fromList [(&quot;bytes&quot;,Number 87490.0),(&quot;numTxs&quot;,Number 152.0)])),(&quot;kind&quot;,String &quot;TraceMempoolRejectedTx&quot;),(&quot;err&quot;,Object (fromList [(&quot;badInputs&quot;,Array [String &quot;d22db824b458406b322b8e2620768bd3c5ff1120208de2fbce57d5b553a69ff5#1&quot;]),(&quot;consumed&quot;,Object (fromList [(&quot;lovelace&quot;,Number 0.0),(&quot;policies&quot;,Object (fromList []))])),(&quot;error&quot;,String &quot;This transaction consumed Value 0 (fromList []) but produced Value 284069674 (fromList [(PolicyID {policyID = ScriptHash \&quot;defca4b95934aec82b818cbafd4bd0ab45511c1d690b58759f07d9b1\&quot;},fromList [(\&quot;VyFi\&quot;,24900)])])&quot;),(&quot;kind&quot;,String &quot;ValueNotConservedUTxO&quot;),(&quot;produced&quot;,Object (fromList [(&quot;lovelace&quot;,Number 2.84069674e8),(&quot;policies&quot;,Object (fromList [(&quot;defca4b95934aec82b818cbafd4bd0ab45511c1d690b58759f07d9b1&quot;,Object (fromList [(&quot;56794669&quot;,Number 24900.0)]))]))]))]))]
--
Jun 17 14:06:07 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Info:102] [2021-06-17 13:06:07.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Skipping peer 192.168.0.12:6000
Jun 17 14:06:07 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:102] [2021-06-17 13:06:07.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Failed to start all required subscriptions
Jun 17 14:06:08 wladyslawa bash[1021152]: [wladysla:cardano.node.LeadershipCheck:Info:92] [2021-06-17 13:06:08.00 UTC] {&quot;kind&quot;:&quot;TraceStartLeadershipCheck&quot;,&quot;chainDensity&quot;:5.040153e-2,&quot;slot&quot;:32368877,&quot;delegMapSize&quot;:644495,&quot;utxoSize&quot;:2660038,&quot;credentials&quot;:&quot;Cardano&quot;}
--
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 15:39:29.00 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceForgedBlock&quot;),(&quot;slot&quot;,Number 3.2378078e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.ChainDB:Error:82] [2021-06-17 15:39:29.01 UTC] Invalid block 95def156e23112ebe3f9c6832ac96059d10ff5c446bdbbbc275b2698dfc336d0 at slot 32378078: ExtValidationErrorHeader (HeaderProtocolError (HardForkValidationErrFromEra S (S (S (Z (WrapValidationErr {unwrapValidationErr = ChainTransitionError [OverlayFailure (OcertFailure (InvalidKesSignatureOCERT 249 243 6 &quot;Reject&quot;))]}))))))
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.ChainDB:Info:82] [2021-06-17 15:39:29.01 UTC] Valid candidate 82548006ab042d802e1f885e028ea93038282d028b9530999e9dfc63473ff247 at slot 32378076
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Error:92] [2021-06-17 15:39:29.01 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceForgedInvalidBlock&quot;),(&quot;reason&quot;,Object (fromList [(&quot;error&quot;,Object (fromList [(&quot;error&quot;,Object (fromList [(&quot;failures&quot;,Array [Object (fromList [(&quot;error&quot;,String &quot;Reject&quot;),(&quot;kind&quot;,String &quot;InvalidKesSignatureOCERT&quot;),(&quot;opCertExpectedKESEvolutions&quot;,String &quot;6&quot;),(&quot;opCertKESCurrentPeriod&quot;,String &quot;249&quot;),(&quot;opCertKESStartPeriod&quot;,String &quot;243&quot;)])]),(&quot;kind&quot;,String &quot;ChainTransitionError&quot;)])),(&quot;kind&quot;,String &quot;HeaderProtocolError&quot;)])),(&quot;kind&quot;,String &quot;ValidationError&quot;)])),(&quot;slot&quot;,Number 3.2378078e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
Jun 17 16:39:30 wladyslawa bash[1021152]: [wladysla:cardano.node.LeadershipCheck:Info:92] [2021-06-17 15:39:30.00 UTC] {&quot;kind&quot;:&quot;TraceStartLeadershipCheck&quot;,&quot;chainDensity&quot;:5.0332207e-2,&quot;slot&quot;:32378079,&quot;delegMapSize&quot;:644766,&quot;utxoSize&quot;:2661388,&quot;credentials&quot;:&quot;Cardano&quot;}
</code></pre>
<h3>All <a href="https://www.dropbox.com/s/gekfu7w7fz05ocj/logs.txt?dl=0" rel="nofollow noreferrer">logs</a> at Dropbox.</h3>
<h3>Update:</h3>
<p>I followed the directions to make sure that the <code>opCertExpectedKESEvolutions</code> matched the actual evolutions.
The command <code>cardano-cli text-view decode-cbor --in-file &lt;path&gt;/op.cert  | grep int | head -1</code> returns <code>6</code> but now the expected evolutions is <code>8</code>???</p>
<p>That did not work. I am getting the same error again.</p>
<pre><code>Jul 01 16:02:50 wladyslawa bash[680275]: [wladysla:cardano.node.Forge:Error:115] [2021-07-01 15:02:50.01 UTC] fromList [(&quot;val&quot;,Object (fromList [(&quot;kind&quot;,String &quot;TraceForgedInvalidBlock&quot;),(&quot;reason&quot;,Object (fromList [(&quot;error&quot;,Object (fromList [(&quot;error&quot;,Object (fromList [(&quot;failures&quot;,Array [Object (fromList [(&quot;error&quot;,String &quot;Reject&quot;),(&quot;kind&quot;,String &quot;InvalidKesSignatureOCERT&quot;),(&quot;opCertExpectedKESEvolutions&quot;,String &quot;8&quot;),(&quot;opCertKESCurrentPeriod&quot;,String &quot;259&quot;),(&quot;opCertKESStartPeriod&quot;,String &quot;251&quot;)])]),(&quot;kind&quot;,String &quot;ChainTransitionError&quot;)])),(&quot;kind&quot;,String &quot;HeaderProtocolError&quot;)])),(&quot;kind&quot;,String &quot;ValidationError&quot;)])),(&quot;slot&quot;,Number 3.3585479e7)])),(&quot;credentials&quot;,String &quot;Cardano&quot;)]
</code></pre>

<stake-pool>
Invalid KES signature OCERT but gLiveView shows KES is valid
# Invalid KES signature OCERT but gLiveView shows KES is valid

(Asked by: username_0 on 2021-06-17)

Have recently rotated KES following steps.
I cannot get any block to get adopted despite being selected as a slot leader number of times already. As you can see Guild LiveView tool shows that KES kes are valid. But the logs from cardano-node show otherwise.


Jun 17 16:39:29 wladyslawa bash[1021152]:


[wladysla:cardano.node.ChainDB:Error:82] [2021-06-17 15:39:29.01 UTC]
Invalid block
95def156e23112ebe3f9c6832ac96059d10ff5c446bdbbbc275b2698dfc336d0 at
slot 32378078: ExtValidationErrorHeader (HeaderProtocolError
(HardForkValidationErrFromEra S (S (S (Z (WrapValidationErr
{unwrapValidationErr = ChainTransitionError [OverlayFailure
(OcertFailure (InvalidKesSignatureOCERT 249 243 6 "Reject"))]}))))))

More errors:
maciej@ola:~/Downloads$ grep -ir ":error" logs.txt  -C 1
Jun 17 14:00:38 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 13:00:38.00 UTC] fromList [("val",Object (fromList [("kind",String "TraceNodeNotLeader"),("slot",Number 3.2368547e7)])),("credentials",String "Cardano")]
Jun 17 14:00:38 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:73120] [2021-06-17 13:00:38.48 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Application Exception: 192.168.0.12:6000 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
Jun 17 14:00:38 wladyslawa bash[1021152]: [wladysla:cardano.node.ErrorPolicy:Notice:100] [2021-06-17 13:00:38.48 UTC] IP 192.168.0.12:6000 ErrorPolicySuspendConsumer (Just (ApplicationExceptionTrace ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply))) 20s
--
Jun 17 14:04:25 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 13:04:25.00 UTC] fromList [("val",Object (fromList [("kind",String "TraceNodeNotLeader"),("slot",Number 3.2368774e7)])),("credentials",String "Cardano")]
Jun 17 14:04:25 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:99511] [2021-06-17 13:04:25.20 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Application Exception: 192.168.0.12:6000 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
Jun 17 14:04:25 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Info:99511] [2021-06-17 13:04:25.20 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Closed socket to 192.168.0.12:6000
--
Jun 17 14:05:56 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 13:05:56.00 UTC] fromList [("val",Object (fromList [("kind",String "TraceNodeNotLeader"),("slot",Number 3.2368865e7)])),("credentials",String "Cardano")]
Jun 17 14:05:56 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:99547] [2021-06-17 13:05:56.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Application Exception: 192.168.0.12:6000 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
Jun 17 14:05:56 wladyslawa bash[1021152]: [wladysla:cardano.node.ErrorPolicy:Notice:100] [2021-06-17 13:05:56.21 UTC] IP 192.168.0.12:6000 ErrorPolicySuspendConsumer (Just (ApplicationExceptionTrace ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply))) 20s
--
Jun 17 14:05:57 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Info:102] [2021-06-17 13:05:57.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Skipping peer 192.168.0.12:6000
Jun 17 14:05:57 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:102] [2021-06-17 13:05:57.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Failed to start all required subscriptions
Jun 17 14:05:57 wladyslawa bash[1021152]: [wladysla:cardano.node.Mempool:Info:99577] [2021-06-17 13:05:57.23 UTC] fromList [("tx",Object (fromList [("txid",String "txid: TxId {_unTxId = SafeHash \"fb5f593592735aa5507b2881705e902611729620968b4c890718cb775ce9d4fc\"}")])),("mempoolSize",Object (fromList [("bytes",Number 87490.0),("numTxs",Number 152.0)])),("kind",String "TraceMempoolRejectedTx"),("err",Object (fromList [("badInputs",Array [String "d22db824b458406b322b8e2620768bd3c5ff1120208de2fbce57d5b553a69ff5#1"]),("consumed",Object (fromList [("lovelace",Number 0.0),("policies",Object (fromList []))])),("error",String "This transaction consumed Value 0 (fromList []) but produced Value 284069674 (fromList [(PolicyID {policyID = ScriptHash \"defca4b95934aec82b818cbafd4bd0ab45511c1d690b58759f07d9b1\"},fromList [(\"VyFi\",24900)])])"),("kind",String "ValueNotConservedUTxO"),("produced",Object (fromList [("lovelace",Number 2.84069674e8),("policies",Object (fromList [("defca4b95934aec82b818cbafd4bd0ab45511c1d690b58759f07d9b1",Object (fromList [("56794669",Number 24900.0)]))]))]))]))]
--
Jun 17 14:06:07 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Info:102] [2021-06-17 13:06:07.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Skipping peer 192.168.0.12:6000
Jun 17 14:06:07 wladyslawa bash[1021152]: [wladysla:cardano.node.IpSubscription:Error:102] [2021-06-17 13:06:07.21 UTC] IPs: 0.0.0.0:0 [192.168.0.12:6000] Failed to start all required subscriptions
Jun 17 14:06:08 wladyslawa bash[1021152]: [wladysla:cardano.node.LeadershipCheck:Info:92] [2021-06-17 13:06:08.00 UTC] {"kind":"TraceStartLeadershipCheck","chainDensity":5.040153e-2,"slot":32368877,"delegMapSize":644495,"utxoSize":2660038,"credentials":"Cardano"}
--
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Info:92] [2021-06-17 15:39:29.00 UTC] fromList [("val",Object (fromList [("kind",String "TraceForgedBlock"),("slot",Number 3.2378078e7)])),("credentials",String "Cardano")]
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.ChainDB:Error:82] [2021-06-17 15:39:29.01 UTC] Invalid block 95def156e23112ebe3f9c6832ac96059d10ff5c446bdbbbc275b2698dfc336d0 at slot 32378078: ExtValidationErrorHeader (HeaderProtocolError (HardForkValidationErrFromEra S (S (S (Z (WrapValidationErr {unwrapValidationErr = ChainTransitionError [OverlayFailure (OcertFailure (InvalidKesSignatureOCERT 249 243 6 "Reject"))]}))))))
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.ChainDB:Info:82] [2021-06-17 15:39:29.01 UTC] Valid candidate 82548006ab042d802e1f885e028ea93038282d028b9530999e9dfc63473ff247 at slot 32378076
Jun 17 16:39:29 wladyslawa bash[1021152]: [wladysla:cardano.node.Forge:Error:92] [2021-06-17 15:39:29.01 UTC] fromList [("val",Object (fromList [("kind",String "TraceForgedInvalidBlock"),("reason",Object (fromList [("error",Object (fromList [("error",Object (fromList [("failures",Array [Object (fromList [("error",String "Reject"),("kind",String "InvalidKesSignatureOCERT"),("opCertExpectedKESEvolutions",String "6"),("opCertKESCurrentPeriod",String "249"),("opCertKESStartPeriod",String "243")])]),("kind",String "ChainTransitionError")])),("kind",String "HeaderProtocolError")])),("kind",String "ValidationError")])),("slot",Number 3.2378078e7)])),("credentials",String "Cardano")]
Jun 17 16:39:30 wladyslawa bash[1021152]: [wladysla:cardano.node.LeadershipCheck:Info:92] [2021-06-17 15:39:30.00 UTC] {"kind":"TraceStartLeadershipCheck","chainDensity":5.0332207e-2,"slot":32378079,"delegMapSize":644766,"utxoSize":2661388,"credentials":"Cardano"}

All logs at Dropbox.
Update:
I followed the directions to make sure that the opCertExpectedKESEvolutions matched the actual evolutions.
The command cardano-cli text-view decode-cbor --in-file /op.cert  | grep int | head -1 returns 6 but now the expected evolutions is 8???
That did not work. I am getting the same error again.
Jul 01 16:02:50 wladyslawa bash[680275]: [wladysla:cardano.node.Forge:Error:115] [2021-07-01 15:02:50.01 UTC] fromList [("val",Object (fromList [("kind",String "TraceForgedInvalidBlock"),("reason",Object (fromList [("error",Object (fromList [("error",Object (fromList [("failures",Array [Object (fromList [("error",String "Reject"),("kind",String "InvalidKesSignatureOCERT"),("opCertExpectedKESEvolutions",String "8"),("opCertKESCurrentPeriod",String "259"),("opCertKESStartPeriod",String "251")])]),("kind",String "ChainTransitionError")])),("kind",String "HeaderProtocolError")])),("kind",String "ValidationError")])),("slot",Number 3.3585479e7)])),("credentials",String "Cardano")]

**Tags:** stake-pool

---

## 3 answers

### Answer by username_1 on 2021-06-18. Score: 5 | Accepted answer

gLiveView can only tell you what your current KES key's start/expiry time is. But another base property of KES key is the number of times your key was rotated. You can check your current key using `cardano-cli text-view decode-cbor --in-file /op.cert | grep int | head -1`. Your current cert can only be higher, (not lower) than the one used for your previous blocks.
The upcoming public pgrest API on guild-operators website (currently in alpha stage) will allow you to query your last created block's OCert number for validation too.

PS: You can rotate multiple times to a higher counter (so for example, you can go rotate until counter is 10) just fine, as it will still be higher than your previous block's OCert counter.

> **Comments:**
> **username_0**: Thank you. That number shows 2 which is what I would expect given that I rotated KES twice already. Would you mind clarify further because I cannot see how I can use this number for further debugging.  
> **username_1**: As per your original thread, it lookslike the counter expected is 6 , not 2. So there is block on chain where your pool's opcert counter was 5 already, and you need to rotate it 4 more times for your index to be higher.  
> **username_2**: I think you can also just open the counter file and manually increase it to something higher.  
> **username_0**: My pooled was elected a block leader yesterday. Unfortunately that changes that you proposed and I enacted did not work. Same error, though expected KES evolutions is now 8.  
> **username_1**: Please specify both your current counter, new counter in opcert and whether your new KES is generated from same cold key  

---

### Answer by username_3 on 2021-06-23. Score: 1

We are seeing similar error. We only have rotated the cert once and that was after minting our first block.
@RdLrT this command shows just 1.
cardano-cli.sh "text-view decode-cbor --in-file node.cert"  | grep int | head -1
      01  # int(1)

But the error says
"opCertExpectedKESEvolutions",String "17"
Does this mean I have to rotate it 16 more times?
cardano-node    | [93f57906:cardano.node.ChainDB:Info:537] [2021-06-17 11:48:08.00 UTC] Valid candidate 1ef53e05e0c8ac9a35519dedc61e974036cb66fbf15a6ecc4c3420fa0db9f8ef at slot 32364102
cardano-node    | [93f57906:cardano.node.Forge:Error:547] [2021-06-17 11:48:08.01 UTC] fromList [("val",Object (fromList [("kind",String "TraceForgedInvalidBlock"),("reason",Object (fromList [("error",Object (fromList [("error",Object (fromList [("failures",Array [Object (fromList [("error",String "Reject"),("kind",String "InvalidKesSignatureOCERT"),("opCertExpectedKESEvolutions",String "17"),("opCertKESCurrentPeriod",String "249"),("opCertKESStartPeriod",String "232")])]),("kind",String "ChainTransitionError")])),("kind",String "HeaderProtocolError")])),("kind",String "ValidationError")])),("slot",Number 3.2364197e7)])),("credentials",String "Cardano")]
cardano-node    | [93f57906:cardano.node.LeadershipCheck:Info:547] [2021-06-17 11:48:09.00 UTC] {"kind":"TraceStartLeadershipCheck","chainDensity":4.991215e-2,"slot":32364198,"delegMapSize":644407,"utxoSize":2659418,"credentials":"Cardano"}

> **Comments:**
> **username_0**: I followed that advice but there is no way to verify if it solves the problem before any blocks are actually minted. I should know the answer within next week or two.  
> **username_4**: This appears to be a new question, not really an answer to the original question.  Please consider whether you should make it a separate question, update or delete it.  Welcome aboard, I hope my feedback is helpful.  
> **username_3**: @gRebel the problem is exactly the same and didn't want to open a new thread on this to avoid missed context.  
> **username_1**: @user2107938 Based on the log message, yes (or you can verify in your dbsync instance too)  

---

### Answer by username_2 on 2021-06-18. Score: 0

If it's not the counter issue mentioned above, I had a similar error in the past:

> [cardano-:cardano.node.Forge:Error:195] [2021-03-15 23:36:47.00 UTC] fromList [("credentials",String "Cardano"),("val",Object (fromList [("kind",String "TraceForgedInvalidBlock"),("reason",Object (fromList [("kind",String "ValidationError"),("error",Object (fromList [("kind",String "HeaderProtocolError"),("error",Object (fromList [("kind",String "ChainTransitionError"),("failures",Array [Object (fromList [("kind",String "InvalidKesSignatureOCERT"),("error",String "Reject"),("opCertKESCurrentPeriod",String "165"),("opCertExpectedKESEvolutions",String "1"),("opCertKESStartPeriod",String "164")])])]))]))])),("slot",Number 2.1482191e7)]))]

I believe the cause was that I only copied the updated `kes` file to the producer, and not the new `node.cert` file. I re-copied both files over to the producer and restarted, and the blocks started working correctly.

> **Comments:**
> **username_0**: Thank you. May I ask you. The steps that I followed only mention about copying node.cert to the prodcution environment.  https://www.coincashew.com/coins/overview-ada/guide-how-to-build-a-haskell-stakepool-node#18-1-rotate-pools-kes-keys-updating-the-operational-cert-with-a-new-kes-period  What steps did you follow? Did you manage to find any officals Cardano docs for this?  
> **username_1**: @matcheek - The issue is already proven to be as per first reply based on log extract and your first reply about current ocert number. It would best for you to rotate your KES 4 more times (so that current is 6 or above) and verify next block assignment is created correctly.  
> **username_0**: @RdLrT Done. Do you know how to verify if KES is correct now before waiting for another minted block?  
> **username_2**: I've no idea why it only mentions one, I'm pretty certain you need to copy both! I followed the instructions on the official Cardano site (which has just been updated and is hard to find things now), and I just repeated the bit I assumed was required for KES rotation. I don't know of any way to verify things are correct before minting a block though (I hit this issue on testnet where I had a large delegation and was able to verify fairly quickly).  

---

