Migration to Streams Server 2021.2 from an Older version

These instructions assume that you:

q  have an existing 2020.1 or older server installed and want to migrate the content to a new installation of the 2020.3 server.

q  want to keep running the old server while you make sure that the migration was successful, and that the new server is running as it should.

All of the server content is stored in its application data <appdata> folder, the path of which is set in the PanopticonAppData context environment property. For example, in Tomcat this would be in <tomcat_home>/conf/catalina/localhost/streams.xml or similar.

 

NOTE	Two Panopticon web applications should never share the same <appdata> folder, ensure that the new server is pointed at its own initially empty folder.

 

Some of the content can simply be copied from the old server to the new one, while some is now stored in a new format and needs to be converted. The applications and data sources themselves can be migrated any number of times, essentially resetting the applications on the new server.

 

1.     Copy All Content

Start by copying all files from <old_appdata>  to <new_appdata>. You can selectively copy some files again later to keep the old and new server in sync (e.g., copy over scheduled tasks after they are modified on the old server). This completes the migration of the following:

q  License file - The server will not start without a valid <appdata>/PanopticonLicense.xml. In 2020 you also have the option to use Altair units licensing instead of the XML file.

q  Properties file - The set of properties in <appdata>/Streams.properties  that the server understands changes between releases. The first time you start it, it will add new one and remove old properties.

q  Scheduled tasks - All scheduled tasks are in SCH files in <appdata>/Schedule/.

 

2.     Delete Old Content

On the new server, delete the <new_appdata>/Tokens/  folder. This holds authentication tokens for logged in users, and they are server specific.

 

3.     One-time Conversion

NOTE	Converting applications and data sources is covered in the next section.

 

On the old server, parameters were stored in <old_appdata>/DefaultParameters.xml. They were global and applied to all content (applications and data sources). In 2021.0, you can now organize content in folders, and you can also define parameters that only apply to content in a particular folder. The new server stores them all in <new_appdata>/Parameters.json.

If <appdata>/Parameters.json  doesn't exist when the new server starts, it will create it, and if it finds <appdata>/DefaultParameters.xml  it will import these into the new file. To repeat the conversion, e.g., if you want to re-import changed parameters from the old server, delete Parameters.json  and restart the server.

 

4.     Applications, Data Sources, and Data files

Applications and their change history, and data sources, are stored in a very different format in a repository inside the <appdata>/.streams-repository/ folder. This is preparation for better versioning, content synchronization in a cluster and other things.

Before version 2020.2, all applications were stored as individual APP files in <appdata>/CEP/Applications. Every time an application was updated, a backup was placed in <appdata>/CEP/Archive. Data sources were stored as DSM files in <appdata>/CEP/Datasources.

If the new server starts and the <appdata>/.streams-repository/ folder doesn't exist, it will create one, and then look in the <appdata>/CEP/ folder. Any applications and data source files it finds in there, it will import into the newly created repository. After the import, the <appdata>/CEP/ folder is no longer used.

Optionally, you can also import all application backups from <appdata>/CEP/Archive/. If you do, they will be recorded as application edits in the new repository. While the web UI currently doesn't expose the change history, it may very well do so in the future.

 

NOTE	To opt out, set repository.import.archived.applications to false in Streams.properties.

 

You can repeat this migration as many times as you like: stop the new server, delete the entire <new_appdata>/.streams-repository/ folder, then start the new server. This provides a convenient way to keep the new server in sync with changes on the old server, assuming the old server is still in use. Please note that this process will lose all changes made on the new server only, as they are stored in the repository.

Data sources that use data files (e.g., CSV, JSON, XML) with relative paths expect the path to be relative to the <appdata>/Data/  folder. You can simply copy the entire <old_appdata>/Data/  folder to <new_appdata>.

 

5.    Do Not Make Changes on Both Servers

After the initial migration you can keep the new server up to date when content changes on the old server by repeating any of the steps above. It is much harder to move content the other way, from the new server to the old one. Therefore, avoid making changes (that you want to keep) on the new server until you've completely migrated and retired the old server.

 

6.    Post-migration Cleanup

When you are satisfied that new server is running as it should, that all content has been migrated, switched users over to the new server, and are no longer using the old server, you can remove files from <new_appdata> that are no longer needed.

q  <appdata>/DefaultParameters.xml  - These are now in the JSON file.

q  <appdata>/CEP/Applications/  - Applications are now stored in the repository.

q  <appdata>/CEP/Archive/  - If you migrated the change history, this is also in the repository now. Otherwise you can keep it if you want to go back to an earlier application version.

q  <appdata>/CEP/Datasources/ - Data sources are now also in the repository.