Databases
In this article:
Databases#
General information#
NGN Cloud databases are automated services which help you to quickly and easily create, configure, and manage popular DBMSs, using cloud platform tools. We currently support MongoDB, MySQL, PostgreSQL, and Redis.
MySQL is the most popular open-source relational DBMS. To learn more about MySQL DBMS capabilities, read the documentation.
PostgreSQL is a Postgres-based object-relational DBMS. It supports most of the SQL standard and offers many advanced features. To learn more about its capabilities, read the PostgreSQL documentation.
Redis is an open-source resident DBMS. It handles key-value data structures and can be used as a database, cache, message broker, or queue. To learn more about Redis capabilities, visit the official website.
MongoDB is a universal NoSQL database that stores data of any structure and supports indexing, sharding, and replication. To learn more about MongoDB features, see the MongoDB official website.
Databases in NGN Cloud#
Using NGN Cloud automated databases is a powerful alternative to installing and configuring databases manually. The database service can be deployed on a single instance or as a cluster of three instances.
The cluster service option ensures high availability for critical workloads by placing nodes in multiple availability zones. If any of these nodes fails, the cluster will continue to run on the remaining nodes. This prevents data loss, and there is no interruption in service, or it is reduced to a minimum delay required for failover.
In the Databases service web interface, you can perform the following actions:
What is the Database service good for?
For quick creation of a ready-to-work database.
For simplifying database provisioning and maintenance.
For accelerating time-to-market.
Cluster with an arbitrator#
If your cloud has three Availability Zones, then you can launch a high-availability service in a three-node cluster. This solves the split brain problem.
If you use a database cluster and connectivity between nodes is lost, the cluster gets partitioned. Each part of the cluster decides whether to continue processing data or stop. If each part of the cluster considers itself the only survivor and continues running and changing data, then there is the split-brain problem that can cause data corruption. To avoid this, clusters use a quorum rule stating that only one part of the cluster (the one with a larger number of nodes) continues operating. This is only possible if the total number of nodes in the cluster is odd.
However, running three full-fledged nodes is redundant in terms of resources involved. Therefore, a cluster with two full-featured nodes and an arbitrator is more cost-effective, since the arbitrator neither stores nor processes data and uses minimum resources to maintain a cluster forum.
Before you begin#
To get started with the Databases service, follow these steps:
Create a project, if you don’t have one.
In the IAM section, create a user with the PaaS Administrator or Cloud Administrator role and add this user to the project with the PaaS privilege. If you want to back up databases, then additionally create a special user with the PaaS Backup User role and add it to the project.
Make sure that your project has all the necessary resources: subnets, SSH keys, security groups, and buckets for backups. If these resources do not exist, create them.
Read the recommendations on how to manage specific databases in the cloud.
Creating database service#
The database service creation process can be divided into two parts:
infrastructure creation;
software installation and configuration.
Currently, the infrastructure creation consists of launching the required number of instances with the specified configuration from a given image and setting up security groups so that the databases can connect to each other.
After the successful launch of the instances, software installation inside VMs begins. Upon the successful installation, the database service enters the running status. Any other status means that the creation process is not completed. The database service in the running status is ready for operation.
To configure the service, in Service Store or in Installed services select the DBMS, click Create and follow the steps below:
Specify the network parameters required for the operation of a standalone or high-availability database service:
VPC where the database service will be deployed.
Security groups, which control traffic of instances interfaces.
Subnets where the database service will be deployed, or network interfaces through which cluster nodes will connect to subnets.
The High-availability service option. If you select this option, then the service will be deployed in a three-node cluster.
Note
To run a high-availability service in the selected VPC, create subnets or network interfaces in each availability zone first. In addition, these availability zones must support the same volume types.
Note
The ability to attach network interfaces may be useful, for example, when you need to recreate a database service. If you delete the service, but do not delete attached network interfaces, you will be able to reuse them for connecting nodes of the new cluster to subnets, where the new cluster will be deployed. Thus, you can keep previous network settings, such as private IP addresses and security groups, rather than configure them again.
Specify the configuration of the instances where the database nodes will run. Select the instance type and parameters of its volumes: type, size and IOPS (if available for the type you choose).
For MySQL, PostgreSQL and MongoDB DBMSs, you can set a configuration with cluster arbitrator. To enable it, select the Use arbitrator option.
Note
Database performance depends on the selection of node components. We recommend using high performance volumes gp2: Universal (SSD) or io2: Maximum (SSD).
In addition, you can also specify an SSH key. In this case, after automatic service setup, you will get an SSH access to the respective instances.
Attention
We provide the option to connect to instances using an SSH key while the new database service is beta testing. This feature may be disabled in the future.
Set the main parameters of the service (the specific set of options depends on the selected service):
Service — An arbitrary unique name of the database service.
Vendor — MySQL DBMS engine. Valid values: mariadb, percona, mysql.
Version — The DBMS engine version. The version depends on the engine vendor and has the following values:
mariadb: 10.2, 10.3, 10.4, 10.5, 10.6, 10.7
percona: 5.7, 8.0
mysql: 5.7, 8.0
Enable monitoring – For centralized PaaS service monitoring, first deploy the Prometheus-based monitoring service. When checking this option, also select Monitoring service you want to use. Optionally, you can set Monitoring labels, which the installed monitoring agents will assign to collected metrics (for details, see about label usage).
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Service — An arbitrary unique name of the database service.
Version — The DBMS version. Valid values: 10, 11, 12, 13, 14, 15
Replication option — The replication mode in the Patroni cluster. Available only when the High-availability service option is selected.
Valid values: asynchronous, synchronous, synchronous_strict.
Enable monitoring – For centralized PaaS service monitoring, first deploy the Prometheus-based monitoring service. When checking this option, also select Monitoring service you want to use. Both services must be in the same VPC. Optionally, you can set Monitoring labels, which the installed monitoring agents will assign to collected metrics (for details, see about label usage).
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Service — An arbitrary unique name of the database service.
Version — The DBMS version. Valid values: 3.6, 4.0, 4.2, 4.4, 5.0
Enable monitoring – For centralized PaaS service monitoring, first deploy the Prometheus-based monitoring service. When checking this option, also select Monitoring service you want to use. Optionally, you can set Monitoring labels, which the installed monitoring agents will assign to collected metrics (for details, see about label usage).
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Service — An arbitrary unique name of the database service.
Version — The DBMS version. Valid values: 5, 6, 7.
Clustering option. Valid values: native, sentinel. Available only when the High-availability service option is selected.
RDB Persistence — The RDB storage mode.
AOF Persistence — The AOF storage mode.
Enable monitoring – For centralized PaaS service monitoring, first deploy the Prometheus-based monitoring service. When checking this option, also select Monitoring service you want to use. Optionally, you can set Monitoring labels, which the installed monitoring agents will assign to collected metrics (for details, see about label usage).
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Password — User password 8 - 128 characters long. It can be set manually or generated automatically.
Configure backup.
Note
This step is currently available for MySQL and PostgreSQL only.
When you select Enable backup, the following settings are available:
Bucket where the backups will be stored.
User with PaaS Backup User rights; backups will be written to the bucket under this user.
Daily backup start time.
Backup retention period.
Email (optional) – the email address to which notifications will be sent if the backup fails. If not specified, notifications will be sent to the email address of the user you have selected above.
Specify advanced settings, that will become part of the DBMS configuration (optional). The supported parameters depend on the type of database service you launch. You can read more about supported parameters in the recommendations for a particular database.
You can also specify advanced settings of your choice at this step. To do this, click advanced settings.
Important
The specified settings will be part of the DBMS configuration and, therefore, will affect its operation. Add only the settings you really need.
After completing all the steps, click Create.
Note
The database creation process usually takes 5 to 15 minutes. Standalone installations are launched faster than high-availability ones.
To ensure the correct operation of a database service, a new security group is automatically created together with the cluster. It includes the following rules:
the rule to permit inbound traffic from interfaces that are in the same security group;
the rule to enable all outbound IPv4 traffic.
If the database service is deleted, the security group will also be deleted.
User and database management#
You can create and delete databases and users, as well as edit some of their settings, directly from the cloud web interface.
Creating and deleting users and databases is only possible for services in the Running or Updating state. These actions are available for MySQL, PostgreSQL and MongoDB databases.
Create user#
To create a new user:
Go to PaaS > Installed Services.
Find the desired service in the table and go to the service page.
Open the Users tab and click Create.
Set the required parameters:
Name — User name. It should start with a Latin letter, should not exceed 32 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as root, mysql, monitoring, mariadb.sys, clustercheck.
Host – name of the host from which the user is allowed to connect to the database. For MySQL, you can also specify the IP address of the host from which the user is allowed to connect to the database. If this field is left blank, the user will be allowed to connect from any IP address and the default value (%) will be set as the host.
Password – User password.
Name — User name. It should start with a Latin letter, should not exceed 31 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as os_admin, postgres, root, replicator.
Password – User password.
Name — User name. It should start with a Latin letter, should not exceed 63 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as admin, root.
Password – User password.
Click Create.
Create a database#
To create a new database:
Go to PaaS > Installed Services.
Find the desired service in the table and go to the service page.
Open the Databases tab and click Create.
Specify a database name and other parameters. The exact set of parameters depends on the selected database service.
Name — Database name. It should start with a Latin letter, should not exceed 64 characters, can consist of numbers, Latin letters, and $ and “_” characters. You cannot use reserved names, such as information_schema, mysql, performance_schema.
Charset – database encoding.
Collation — Database collation parameters or collation parameter normalization operation.
Enable backup — Enables daily backup. Backups will only be created if this option is enabled for the service.
Name — Database name. It should start with a Latin letter, should not exceed 31 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as postgres, template0, template1.
Owner — The database owner. This must be one of existing users added at the create user step. This user cannot be deleted as long as it remains the database owner.
Locale — The database locale.
Encoding — The database encoding.
Extensions — The extensions to be installed for the database.
Enable backup — Enables daily backup. Backups will only be created if this option is enabled for the service.
Name — Database name. It should start with a Latin letter, should not exceed 63 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use the reserved name admin.
You can grant users access to a database right away or later. To add users later and create a database right away, click Create. To add users, click Edit Users to go to the next step.
Click Add User and select the previously created user from the list. For MySQL database, you can also assign the necessary rights to the user.
Privileges — The user privileges in the database.
Options — Operations that the user can perform on the database.
To add other users, click Add User.
Click Create.
Change user password#
To change a user’s password after creating it, follow these steps:
Go to PaaS > Installed Services.
Find the desired service in the table and go to the service page.
Open the Users tab, select the user whose password you want to change, and click Change.
In the dialog window, set or generate a new password.
Click Save.
Change database settings#
Once database has been created, you can modify some of its settings and create/delete users. To do this:
Go to PaaS > Installed Services.
Find the desired service in the table and go to the service page.
Open the Databases tab, find the database whose settings you want to change, and click Change.
Specify new parameters for the database.
Note
Once a database has been created, its name and language settings cannot be modified. Which parameters you can change depend on the selected database service (MySQL or PostgreSQL).
To allow or deny user access to the database, go to the next step by clicking Edit Users. If no user has been previously granted access to the database, click Add User.
To replace one user with another, select the user from the list.
To deny user access to the database, click the icon next to the user name.
To grant access to the database to additional users, click Add User.
For MySQL and MongoDB users, you can also change their access rights.
Click Save.
The database service switches to the Updating state. Users can continue working with the database as long as the changes do not affect their database access privileges.
Delete user#
To delete a user:
Go to PaaS > Installed Services.
Find the desired service in the table and go to the service page.
Open the Users tab and select the user you want to delete.
Click Delete.
In the dialog window, confirm the deletion.
Note
The user will also be removed from all databases to which he/she was granted access.
Delete database#
To delete a database:
Go to PaaS > Installed Services.
Find the desired service in the table and go to the service page.
Open the Databases tab and select the database you want to delete.
Click Delete.
In the dialog window, confirm the deletion.
Database backup and recovery#
Scheduled backups can be enabled for a database service so that full backups of the specified databases are created daily at the specified time. If necessary, you can quickly restore the desired database as of a specific date from the saved backup.
Note
Backup is now only available for PostgreSQL and MySQL database services. In the future, we plan to deploy it for other database services.
To enable backup, configure the schedule for the database service and enable the appropriate option for each database you want to backup. Backups are only created for running services and are written to the object storage.
To free up space from no-longer-needed backups, we recommend limiting their retention period or regularly deleting them. Certain backups can be protected from automatic deletion.
Along with the backup, a log is created to store information about the backup utility operation. If the backup has failed, you can use the log to find out why. We recommend checking the log for warnings on successful backups also to make sure that there were no problems during the backup process.
Attention
To avoid backup problems caused by lack of space for temporary files, we recommend that you leave 20 MB of free space in the root partitions on cluster instance volumes.
Information about database services for which backups are available can be found in the Service Backups subsection of the PAAS section.
Note
Before configuring backup, make sure that the project has both a bucket for storing backups and a user with PaaS Backup User privileges. This is a special user, and it should only be assigned this role.
Setup service backup#
Backup can be enabled when you create a database service. If you have not done this and want to enable it, or if you need to change the settings for an already enabled backup, then:
Go to PAAS Installed Services and select the DATABASE service type.
Find the desired service in the table and go to the service page.
In the Information tab, click Backup settings.
If backup has not been enabled, check the Enable backup checkbox.
Set or change backup settings:
Bucket where the backups will be stored.
User with PaaS Backup User rights; backups will be written to the bucket under this user.
Daily backup start time.
Backup retention period.
Email (optional) – the email address to which notifications will be sent if the backup fails. If not specified, notifications will be sent to the email address of the user you have selected above.
To confirm, click Change.
Note
To back up specific databases, enable the respective option for each of them.
Enable database backup#
Note
Database backups will only be created if backup for the database service is enabled.
If you want to enable backup for a specific database:
Go to PAAS Installed Services and select the DATABASE service type.
Find the desired service in the table and go to the service page.
Open the Databases tab and select the database in the resource table.
Click Edit.
Check the Enable backup checkbox, while leaving all other parameters unchanged.
Click Save.
Recover a database from a backup#
Important
To avoid problems when recovering the database, we recommend that you recover it to a service with the same DBMS version as that used to create the database.
To recover a database:
Go to PAAS Installed Services and select the DATABASE service type.
Find the desired service in the table and go to the service page.
Open the Databases tab and click Create from backup.
In the database creation wizard that opens, select the required backup from the list. To simplify the search, you can select the service from which originates the database, whose backup you are looking for. Click Select to go to the next step.
Specify the database parameters and, if necessary, add users. These steps are the same as the steps 4-6 when creating a database and are described in more detail in database creation instructions.
Attention
If parameters other than default ones were specified when creating the database, such as PostgreSQL locale, we recommend that you enter the same parameters for the correct recovery.
Disable backup#
To disable backup for a specific database:
Go to PAAS Installed Services and select the DATABASE service type.
Find the desired service in the table and go to the service page.
Open the Databases tab and select the database in the resource table.
Click Edit.
Uncheck the Enable backup checkbox, while leaving all other parameters unchanged.
Click Save.
To disable backup for all databases:
Go to PAAS Installed Services and select the DATABASE service type.
Find the desired service in the table and go to the service page.
In the Information tab, click Backup settings.
Uncheck the Enable backup checkbox.
To confirm, click Change.
Delete backups#
You can delete either specific or all unnecessary backups.
To delete specific backups:
Go to PAAS Service backups.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Click the service ID. It corresponds to the name of the folder in the bucket where database backups for this service are stored.
Open the Backups tab and select the backups for deletion in the resource table.
Click Delete and confirm the action in the dialog window.
Go to PAAS Installed Services.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Go to the service page and open the Backups tab.
In the resource table, select the backups to be deleted.
Click Delete and confirm the action in the dialog window.
To delete all backups:
Go to PAAS Service backups.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Click Delete service backups and confirm the action in the dialog window.
Go to PAAS Installed Services.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Go to the service page and open the Backups tab.
Click Delete all backups and confirm the action in the dialog window.
Go to PAAS Service backups.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Click the service ID. It corresponds to the name of the folder in the bucket where database backups for this service are stored.
Open the Backups tab.
Click Delete all backups and confirm the action in the dialog window.
Note
When you delete all the backups of the service, the folder that was automatically created in the backet to store them is also deleted.
Automatic backup deletion#
In the backup settings, you can set the backup retention period for the service. By default, backups are retained permanently. You can specify the number of days, weeks, months or even years for how long the created backups should be retained.
Individual backups can be protected from automatic deletion. To do this, find the required backup in the list of backups and turn on the Deletion protection switch.
Note
When you delete a database service, automatic deletion of backups will no longer be performed. Backups of the deleted service will be retained permanently. In this case, no-longer-needed backups will have to be deleted manually.
Deleting database service#
Deleting the database service deletes all instances and volumes created with it.
To delete a service, go to the Installed Services section and click the icon in the table or the Delete button on the service page. If you want to use network interfaces in the future, for example, to recreate a service with the same network parameters, then in the dialog window, clear Delete associated network interfaces checkbox.
Note
When a service is deleted, stored backups of its databases are retained.