Solved

On premise S3 Cloud and DDB data verification

  • 16 March 2022
  • 5 replies
  • 167 views

Userlevel 2
Badge +7

Hi,

i have a question about DDB data verification on a Scality ring S3 cloud (on premise).

We have around 800TB of stored data on S3.

Today the ddb verification move on average 1000 TB every month (1.5X as the volume of data stored on S3 scality).

We have set the DDB data verification option to “Quick Verification of Deduplication Database”

it is possible to reduce the number of GET per month by changing the way DDB verification run?

regards,

 

 

icon

Best answer by christophe 22 April 2022, 09:02

View original

If you have a question or comment, please create a topic

5 replies

Userlevel 6
Badge +15

Good morning.  There is no setting to change the way that the DDB verification runs.  For cloud settings we generally recommend not to run DDB verification unless needed as the process needs to read many files, which is the reason for the high “GETs”.  By “unless needed” I mean if you start to see errors related to issues with the data.  You can also reduce the frequency that DDB verification is run.

Userlevel 2
Badge +7

Good morning.  There is no setting to change the way that the DDB verification runs.  For cloud settings we generally recommend not to run DDB verification unless needed as the process needs to read many files, which is the reason for the high “GETs”.  By “unless needed” I mean if you start to see errors related to issues with the data.  You can also reduce the frequency that DDB verification is run.

Hi @Orazan , Thanks for your reply. 

Do you know what is the API use for a quick verification instead of a full verification (GET ) ?

Regards,

Userlevel 6
Badge +15

@christophe Good afternoon.  I apologize but I do not know what exactly the API used is.  I will see what I can find out.

 

Userlevel 2
Badge +7

@christophe Good afternoon.  I apologize but I do not know what exactly the API used is.  I will see what I can find out.

 

Hi @Orazan , i have the answer for my question with the API use by the quick verification process.

the quick verification perform a HEAD instead of a GET, this reduce a lot the load (data read and network bandwith) on the backend side (scality ring on premise).

We observe on scality ring monitoring a big difference between the two options for Data verification.

Userlevel 6
Badge +15

@christophe Thank you very much for updating with the answer to your question.