PANIC: could not locate a valid checkpoint record (postgresql log)

OpenERP is not starting after restore. Found out that postgresql is not starting. Checked the log in /var/lib/pgsql/9.2/data/pg_log/ which shows

LOG: invalid secondary checkpoint record PANIC: could not locate a valid checkpoint record LOG: startup process (PID 5654) was terminated by signal 6: Aborted LOG: aborting startup due to startup process failure

Further came to know that there could be problem with transaction log. So tried sudo /usr/pgsql-9.2/bin/postgres pg_resetxlog DATADIR Gives error - “root” execution of the PostgreSQL server is not permitted. The server must be started under an unprivileged user ID to prevent possible system security compromise. See the documentation for more information on how to properly start the server.

Anyway out?

Did su - postgress then /usr/pgsql-9.2/bin/pg_resetxlog -f /var/lib/pgsql/9.2/data/

https://bahmni.atlassian.net/wiki/spaces/BAH/pages/33128505/Install+Bahmni+on+CentOS needs update that not all log files are in /var/log postgresql log files are in /var/lib/pgsql/9.2/data/pg_log/

But still could not start pgsql now the log file shows INFO: archive-get command end: aborted with exception [119] WARNING: WAL was generated with wal_level=minimal, data may be missing HINT: This happens if you temporarily set wal_level=minimal without taking a new base backup. FATAL: hot standby is not possible because wal_level was not set to “hot_standby” on the master server HINT: Either set wal_level to “hot_standby” on the master, or turn off hot_standby here. LOG: startup process (PID 5889) exited with exit code 1 LOG: aborting startup due to startup process failure

I did yum remove postgres92 bahmni -i local install and miraculously the server started with all my data restored but in bahmni -i local install it fails to install pgbackrest

without the install log, its difficult to trace/say/suggest anything.

Here is the log if it helps us 2017-09-21 06:59:40,644 p=31134 u=root | TASK [pgbackrest-install : Creating postgres stanza] *************************** 2017-09-21 06:59:40,644 p=31134 u=root | task path: /opt/bahmni-installer/bahmni-playbooks/roles/pgbackrest-install/tasks/main.yml:112 2017-09-21 07:00:42,346 p=31134 u=root | fatal: [localhost]: FAILED! => {“changed”: true, “cmd”: “sudo -u postgres pgbackrest --stanza=bahmni-postgres --log-level-console=info stanza-create”, “delta”: “0:01:01.291583”, “end”: “2017-09-21 07:00:42.289949”, “failed”: true, “rc”: 119, “start”: “2017-09-21 06:59:40.998366”, “stderr”: “”, “stdout”: “2017-09-21 06:59:41.226 P00 INFO: stanza-create command begin 1.12: --db-path=/var/lib/pgsql/9.2/data --log-level-console=info --repo-path=/var/lib/pgbackrest --stanza=bahmni-postgres\n2017-09-21 06:59:42.199 P00 INFO: switch xlog 000000030000000000000078\n2017-09-21 07:00:42.241 P00 ERROR: [119]: WAL segment system-id 6462730428162571221 does not match archive system-id 6467516323106488285\n HINT: are you archiving to the correct stanza?\n2017-09-21 07:00:42.242 P00 WARN: WAL segment 000000030000000000000078 did not reach the archive:\n HINT: Check the archive_command to ensure that all options are correct (especialy --stanza).\n HINT: Check the PostreSQL server log for errors.\n2017-09-21 07:00:42.252 P00 ERROR: [119]: (undefined)\n2017-09-21 07:00:42.252 P00 WARN: the stanza bahmni-postgres could not be created\n2017-09-21 07:00:42.253 P00 INFO: stanza-create command end: aborted with exception [119]”, “stdout_lines”: [“2017-09-21 06:59:41.226 P00 INFO: stanza-create command begin 1.12: --db-path=/var/lib/pgsql/9.2/data --log-level-console=info --repo-path=/var/lib/pgbackrest --stanza=bahmni-postgres”, “2017-09-21 06:59:42.199 P00 INFO: switch xlog 000000030000000000000078”, “2017-09-21 07:00:42.241 P00 ERROR: [119]: WAL segment system-id 6462730428162571221 does not match archive system-id 6467516323106488285”, " HINT: are you archiving to the correct stanza?", “2017-09-21 07:00:42.242 P00 WARN: WAL segment 000000030000000000000078 did not reach the archive:”, " HINT: Check the archive_command to ensure that all options are correct (especialy --stanza).", " HINT: Check the PostreSQL server log for errors.", “2017-09-21 07:00:42.252 P00 ERROR: [119]: (undefined)”, “2017-09-21 07:00:42.252 P00 WARN: the stanza bahmni-postgres could not be created”, “2017-09-21 07:00:42.253 P00 INFO: stanza-create command end: aborted with exception [119]”], “warnings”: [“Consider using ‘become’, ‘become_method’, and ‘become_user’ rather than running sudo”]} 2017-09-21 07:00:42,354 p=31134 u=root | to retry, use: --limit @/opt/bahmni-installer/bahmni-playbooks/all.retry

And the corresponding postgres log

ERROR [119]: WAL segment system-id 6462730428162571221 does not match archive system-id 6467516323106488285 HINT: are you archiving to the correct stanza? LOG: archive command failed with exit code 119 DETAIL: The failed archive command was: pgbackrest --stanza=bahmni-postgres archive-push pg_xlog/000000030000000000000072