Solved

Data aging, deleting old jobs


Userlevel 1
Badge +12

Hello, 

 

We have some issues with getting storage space free. When i run Forecast repoort i dont see sometging wrong, mostly jobs are, basic days or last of the week/month under

On the webconsole under Storage - Data Retantion i see 80TB above year.

 

Really i those data i cannot find under the SP.

Last week under SP> Summory> Storage Policy / Copy Space storage Recovery prediction i sow 16-4 35TB and 30 TB on 17-4. Under are prediction for this week

After aging i sow alot of prunable records in DDB, i run DDB verification and after really didnt sow any space freed on storage. I run Space reclamintation with lvl 1 with Clear orphan data, nothing changes on Storage.  I wil uploade dataging log, if need to see sidbprune log i can upload it aswel

icon

Best answer by Orazan 20 April 2022, 00:30

View original

14 replies

Userlevel 1
Badge +12

@Orazan Is there to see how much data to be free after aging? Examplare i have deleted job which had data write of 18 TB, is this mean that i wil get 18 TB back on the storage?

 

Of if i see in logs FinalizeSFile:4051 Removed [/disklibrary/071c/DT3FUV_03.25.2022_13.53/CV_MAGNETIC/V_2288232/CHUNK_20189856/SFILE_CONTAINER_110].  Can i check how much i get space free of this removing job ? 

Userlevel 7
Badge +23

@Egor Skepko , not exactly, though if you want a ballpark, multiply the size of the job by your dedupe rate and you’ll get an idea. 

It’s less accurate for single jobs and more accurate for multiple jobs.

Userlevel 1
Badge +12

@Mike Struening Ok thanks Mike for the explanation

Userlevel 1
Badge +12

@Orazan Sorry for late reaction about logs, here is the physical and prunable logs of today 

 

 

Userlevel 7
Badge +16

It seems pruning is being performed, but I am curious if you see activity in SIDBPhysicalDeletes.log

This log shows actual deletion on the volume.

 

Userlevel 6
Badge +15

If this data is deduplicated, then there will be no expectation of getting back physical space when deleting jobs.  When a unique block of data is deduplicated, that block is written to protected storage.  Any time that block of data is seen again, the DDB is noted to reference the baseline block.  It will not be until every job that references the original block ages that the physical block will be freed.  

 

You can run a Data Retention and Forecast Compliance report for the library in question and provide a copy for review.  This should tell us why every job is being held in retention and how much would be expected to be freed when data aging runs.

Userlevel 1
Badge +12

@Orazan Do you want that i uploade the Data Retention and Forecast Compliance report of libray here? 

 

On the view selection do you want to check all 3 boxes? 

 

Show prunnable and unprunnable data and show data retantion beyond basic data retantion only

Userlevel 6
Badge +15

Pruning records of deduplicated jobs does not have the expectation of freeing up physical space.  You will only regain physical space if baseline blocks are pruned.  This will only happen when every job that references a block of data ages.  

Is this a cloud or disk library?

Userlevel 6
Badge +15

Yes you can upload a copy of the DRFC here.  I would recommend to choose the library or storage policy that you wish to run the report against and leave the rest at default.

Userlevel 1
Badge +12

@Jos Meijer Here are the log for Sidbprune and only 6-4 was something happening 

 

@Orazan This is disk library

Userlevel 1
Badge +12

@Orazan Ok hier are 2 SP of VM back only

Userlevel 6
Badge +15

Good morning.  Can you please update if you are still seeing an issue?

Userlevel 1
Badge +12

@Orazan Sorry i was away on vacation. Wel in the logs i see that something getting deleted. Its seems working. But the storage not getting free much as i wanted. Seems we have more backup then aging weekly

 

293430 7e3cd  07/11 09:52:42 ### 28-3 FinalizeSFile:4051 Removed [/disklibrary/0701/Z1KG41_Folder1/CV_MAGNETIC/V_2337057/CHUNK_20629735/SFILE_CONTAINER_001].
 

Userlevel 6
Badge +15

Can you upload a copy of SIDBPhysicalDeletes.log for review?

 

Reply