DNA was no longer respondiing HTTP/HTTPS

[angel almaras]

DNA service is active, but I do not have access to the web portal.

support me with the troubleshooting documentation and if there is a way to restart the http process.

[balaji.bandi]

What DNAC version, is this new Setup ?

Do you have access to cli - then check :

maglev package status
maglev catalog package display

magctl appstack status

[angel almaras]

hello 
the version in 2.1.2.6

the error that I get via cli is as follows:

- WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),)': /v2/keys/maglev/config/node-1.1.1.1?sorted=true&recursive=true
- WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),)': /v2/keys/maglev/config/node-1.1.1.1?sorted=true&recursive=true
- WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),)': /v2/keys/maglev/config/node-1.1.1.1?sorted=true&recursive=true
- ERROR:etcd.client:Request to server https://1.1.1.1:4001 failed: MaxRetryError(u"HTTPSConnectionPool(host=u'1.1.1.1', port=4001): Max retries exceeded with url: /v2/keys/maglev/config/node-1.1.1.1?sorted=true&recursive=true (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))",)
- WARNING:root:[Attempt 1] Connection to etcd failed due to MaxRetryError(u"HTTPSConnectionPool(host=u'1.1.1.1', port=4001): Max retries exceeded with url: /v2/keys/maglev/config/node-1.1.1.1?sorted=true&recursive=true (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))",). Retrying in 1 seconds...

unsuccessfully tried to renew certificate
- sudo maglev-config certs info
- sudo maglev-config certs refresh

the output of the commands shows the following:

- $ maglev package status
ERROR: HTTPSConnectionPool(host='kong-frontend.maglev-system.svc.cluster.local', port=443): Max retries exceeded with url: /api/system/v1/catalog/settings?repository=main (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff04bc4edd0>: Failed to establish a new connection: [Errno 111] Connection refused',))

- $ maglev catalog package display
ERROR: HTTPSConnectionPool(host='kong-frontend.maglev-system.svc.cluster.local', port=443): Max retries exceeded with url: /api/system/v1/catalog/release-channel?allVersions=false&repository=main (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f29c3d3ae10>: Failed to establish a new connection: [Errno 111] Connection refused',))

do you have any idea what is going on?

[balaji.bandi]

Time for raise an tac case to investigate for you.

also suggest to upgrade to 2.3.5 onwards there are lot of bug fixed (also check hidden bugs or caveats)

[estetson]

Do you remember whether or not you have a good backup? 2.1.2.x is End of Support since June last year: 

https://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/dna-center/eos-eol-notice-c51-743588.html

In order to receive support from TAC, you may need to re-image your machine and restore from a backup, and then upgrade to a supported release. Currently, this would be 2.3.3.7. The issue you're seeing is typically due to one of our internal certificates expiring. This could be due to a defect (there are a few on 2.1.2.x), or it could be the NTP server that's configured is bad and the Catalyst Center didn't renew the certs when it should have

[Torbjørn]

It seems like you've run into the expiring ETCD certificate issue(FN74065). As @balaji.bandi said, you will have to raise a TAC case to have this resolved.

[maflesch]

Yes, you are hitting what appears to the field notice, which also shows you are not on 2.1.2.x but on at least 2.3.3.x, if not already on 2.3.5.x.

If you issue "_shell" and it asks for a password, you are on 2.3.3.x. If you do that and it attempts to ask for token generation, you are on 2.3.5.x. If you are on 2.3.3.x, just follow the field notice to fix the issue. If you are on 2.3.5.x, it requires TAC assistance to resolve.