View Issue Details

IDProjectCategoryView StatusLast Update
0004823SOGosogo-toolpublic2019-10-07 10:01
Reporterflorian Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status newResolutionopen 
Product Version4.0.8 
Summary0004823: It is not possible to restore user data with sogo-tool on a fresh installation
Description

I made a fresh installation of Sogo 4 on Debian and tried to restore the user data that I backed up with sogo-tool backup from an old installation.

But for every user but one I get the error
sogo-tool[29650:29650] user 'xyz' not found

only for the one user that I used to login and test the website the restore worked.

So it seems like the restore is only possible after a user has logged in.
This is very impractical.

TagsNo tags attached.

Activities

Christian Mack

Christian Mack

2019-09-25 07:35

developer   ~0013768

Did you restore preferences first, then calendars and address books and preferences again?

Does "old installation" refere to a v3 or v2 setup?

florian

florian

2019-09-25 08:16

reporter   ~0013769

I did restore preferences first, then calendars and address books
but not preferences again.
I used the small script mentioned here: http://wiki.sogo.nu/backupRestore

Old installation refers to a v2 setup - 2.3.23 to be precise.

Christian Mack

Christian Mack

2019-09-26 07:48

developer   ~0013774

I updated http://wiki.sogo.nu/backupRestore

You did configure the correct SOGoUserSources before restoration?
Do you use the same authentication source as before?
Did you change to nine tables configuration on the v4?

florian

florian

2019-10-02 13:02

reporter   ~0013797

I tried your updated restore script but get the same errors.

Yes, I did configure SOGoUserSources.
The authentication source is not the same as before but I made sure that the UID is the same for each user.

How do I select tables configuration on a fresh v4 installation with an empty database?
I thought a new installation would always use the nine tables configuration?

The configuration and database works when I do not use sogo-tool for restoration but do a restoration of a pg_dump of the v2 database
and run scripts
sql-update-3.0.0-to-combined.sh
sql-update-3.2.10_to_4.0.0.sh
All data present, calendears, events, adressbooks, subscriptions ... work.

When I do a restoration of a pg_dump of the v2 database
and run only script
sql-update-3.2.10_to_4.0.0.sh
to keep the old database layout, sogo is not working right.
I see my personal calendars, but they are empty and no subscribed calendars are visible.

florian

florian

2019-10-02 13:11

reporter   ~0013798

Maybe this is part of the issue, sogo-tool reports
<0x0x55fafad941c0[SOGoCache]> Using host(s) 'localhost' as server(s)

but the old configuration was using '127.0.0.1' as server address
and I configured the new installation the same way
because I read that the server address is referenced inside the table entries
so I didn't dare to change it.

Christian Mack

Christian Mack

2019-10-02 14:14

developer   ~0013799

Nine tables configuration depends on your configuration in sogo.conf.
If you define OCSStoreURL, OCSAclURL and OCSCacheFolderURL it will use it, else it will use the multiple tables per user schema.
Seems you have defined those.

Does disabling IPv6 support on operating system level help?

florian

florian

2019-10-02 15:17

reporter   ~0013800

Last edited: 2019-10-02 15:18

I understand, I haven't defined OCSStoreURL, OCSAclURL and OCSCacheFolderURL myself but I guess they were created in sogo.conf, when I ran sql-update-3.0.0-to-combined.sh
That explains why the database with the old database layout didn't work afterwards.

Disabling IPv6 support didn't change the behaviour, but I understood now what the actual behaviour is:

Your guess with the SOGoUserSources was the right direction to look at.

sogo-tool does only restore data of users that exist in the SOGoUserSources.
Those that do not exist yet fail with error "user 'xyz' not found"
I thought all user restoration would fail because also for users that do exist in SOGoUserSources I got errors like this:

sogo-tool restore -p sogobackup-20191002 uvw
<0x0x562a22b92890[SOGoCache]> Cache cleanup interval set every 1800.000000 seconds
<0x0x562a22b92890[SOGoCache]> Using host(s) 'localhost' as server(s)
<0x0x562a22b92890[SOGoCache]> an error occurred when caching value for key 'uvw+attributes': "SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY"
<0x0x562a22b92890[SOGoCache]> an error occurred when caching value for key 'uvw@zzz.de+attributes': "CONNECTION FAILURE"
<0x0x562a22b92890[SOGoCache]> an error occurred when caching value for key 'uvw+defaults': "SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY"
<0x0x562a22b92890[SOGoCache]> an error occurred when caching value for key 'uvw+defaults': "CONNECTION FAILURE"
<0x0x562a22b92890[SOGoCache]> an error occurred when caching value for key 'uvw+settings': "SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY"
<0x0x562a22b92890[SOGoCache]> an error occurred when caching value for key 'uvw+settings': "SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY"

But in fact data for those existing users was restored successfully.

I don't know if it is useful that sogo-tool refuses to restore data of users that don't exist in the SOGoUserSources.
From a disaster recovery perspective this is quite inflexible.
When you don't know that you have to restore the LDAP server or other authentication source before you can restore the sogo server data, this can cause quite a headache for a poor sysadmin :)

Christian Mack

Christian Mack

2019-10-04 07:06

developer   ~0013801

If I understand you correctly, you want an option to restore people from backup, that can not be authenticated via SOGoUserSources?
Not sure that is even possible.
But what would be the use case?

Or do you want an option to suppress those error messages for not authenticable accounts?

florian

florian

2019-10-07 10:01

reporter   ~0013805

It's not that I need this option badly, but I would find it useful in a disaster recovery scenario.
So one would not depend on restoring the authentication server before one can restore the sogo user data.

Or you could make it more understandable in the error message what the root cause of the error is.
With the simple error "user 'xyz' not found" I was thinking the error was within the backup and sogo-tool could not find the user in the backup.

Issue History

Date Modified Username Field Change
2019-09-24 13:36 florian New Issue
2019-09-25 07:35 Christian Mack Note Added: 0013768
2019-09-25 08:16 florian Note Added: 0013769
2019-09-26 07:48 Christian Mack Note Added: 0013774
2019-10-02 13:02 florian Note Added: 0013797
2019-10-02 13:11 florian Note Added: 0013798
2019-10-02 14:14 Christian Mack Note Added: 0013799
2019-10-02 15:17 florian Note Added: 0013800
2019-10-02 15:18 florian Note Edited: 0013800
2019-10-04 07:06 Christian Mack Note Added: 0013801
2019-10-07 10:01 florian Note Added: 0013805