Solved

Activate FSO experiences and why am I missing the option to archive


Userlevel 7
Badge +19

I'm investigating if we could blend in Activate FSO to fulfill an archiving use-case, but according to the documentation it seems Activate FSO does not allow you to pick the results and perform an archive action. The documentation only refers to  (https://documentation.commvault.com/11.22/expert/118724_moving_files_in_file_storage_optimization.html) the ability to move, delete or tag the data. Is this correct? And if that is the case then what's the reason that we do not have the ability to perform a file archive? 

icon

Best answer by Patrick McGrath 2 March 2021, 15:50

View original

17 replies

Userlevel 7
Badge +23

Hey @Onno van den Berg !  Thanks for the topic.  

Want to be sure I understand what you’re looking to do before I can answer:

For anyone else reading this, File Storage Optimization (FSO) allows you to analyze unstructured data and optimize for volume, availability, performance, and risk.  After analysis, you can delete duplicate files or unused files. You can also move files to a different storage location. For example, you can move files to less expensive secondary storage.

Pulled from here.

With that said, you are correct that there are 3 operations:

Delete, Move, and Tag.

Are you looking to kick off an Archive operation based on the Tags?  Or are you using Archive and Move interchangeably?

Let me know exactly what you’re looking to do and I’ll loop in anyone else if needed.

Thanks!!

Userlevel 7
Badge +19

Hi @Mike Struening,

Yes, your understanding is indeed correct. It only allows you to delete it or to move the data to a different location, but it does not allow you to archive the data e.g. replace the data with a stub. So the only solution it offers from a functional perspective is to make the data unavailable to the end users / consumers of the data which I do not like because it will make the data not accessible anymore so the first spot to open a ticket for the mis at the the location of the backup admin who than get's the question. 


Hi Mr. Backup,
I was going through my file archive because I need the data from an old file but it is gone. I do not really know the file name anymore but it was a big PDF, with (if I recall correct) something with file-customer in its name. Can you please search it for me…….

Userlevel 7
Badge +23

Makes sense.  Not sure if Archiving is part of the intended use case with FSO, though I can definitely tag in some people!

give me a bit to get some replies for you.

Userlevel 2
Badge +3

Hi Onno!   You read our minds!!

Archiving will be added to File Storage Optimization as a remediation action in feature release 11.23, available later this month.  It presents a way for much tighter integration between the Activate and Complete products, and allow archiving based on all of the filters available on the FSO dashboards.

Best, Patrick

Userlevel 7
Badge +23

Hi Onno!   You read our minds!!

Archiving will be added to File Storage Optimization as a remediation action in feature release 11.23, available later this month.  

13 days to goooo

 

Userlevel 7
Badge +19

Yeah I heard it too! Nice! Good to see that we see some integration taking place now and that it comes just in time ;-)

Now my next question comes……. can we expect another form of archiving technology that uses symbolic links/symlinks/soft links instead of stubs? I see a huge advantage if this would be possible, because it still allows an application to open a file, you do not have to wait for it and it uses functionality that is available as part of the OS so no need for specific drivers. 

The 3dfs feature could be used as the target and would in that case comes with one disadvantage which is that it will be a read-only copy of the file (if I'm correct) but there must be a way to make it available for writes as well ;-)


 

Badge +2

@Onno van den Berg We are reviewing this request and I will keep you posted on the developments. If you are not already aware, we do provide a stubbing option where the files are seamlessly recalled when the user/application accesses it. However, I do understand the additional benefits with the symlinks.  

Userlevel 4
Badge +12

Brilliant.

I’ve had the same need for 12 months and was heading down the route of writing code to create an integration required.

Glad to know we’re only a ‘dot release’ away.

 

Userlevel 4
Badge +12

Patrick, Mike

Will this be limited to FSO, or can files identified through the ediscovery and Compliance become a ‘target’ for Archive 

Userlevel 2
Badge +3

@YYZ 

I think we’re a fraction of a dot away, since I believe 11.23 will be available in a couple of weeks or so, hot out of the oven.

At this point, the archiving integration is limited to FSO.   We will be integrating with DG as well, timing TBD.  

The eDiscovery side will take some thinking on approach, since archiving is usually a live data consideration and eDiscovery focuses on getting trusted, time interval data from backups.   The other consideration here is from a compliance / evidentiary standpoint and related to concerns about “defensible deletion”, where we don't necessarily want to be seen to archiving/disposing of items that could be considered evidence unless there’s a stated policy to do so.   If you want to discuss your scenario in a little more detail, I'd love to chat and can be reached at pmcgrath@commvault.com.

Cheers - Patrick

 

 

Userlevel 7
Badge +19

@Patrick McGrath / @PraveenV I have created CMR 311743 to enhance the existing archiving capabilities to add symlink functionality as an alternative to stubbing. Would be awesome to have this capability as part of Complete and Activate.

Userlevel 4
Badge +12

Symlinks would get my vote.

Great to hear a CMR has been created for consideration

 

 

 

Userlevel 3
Badge +3

Hi @Onno van den Berg ,

now that’s a fascinating idea you have there with the symlinks! What I would like to understand - if you have a symlink on the OS pointing to 3DFS and 3DFS has the archived file. What is the actual benefit of this over a regular Commvault stub?

By the way - nice to “read” you here… we should see each other in person again whenever this crazy pandemic is over :)

Have a great evening ,

Christian

 

Userlevel 7
Badge +19

Hi @Christian Kubik 

Glad to see you around on the community forum! Really looking forward to come to TF to meet you all once again. Let's hope we can get our lives back soon! 

Actual benefit:

  • No need for a proprietary driver, symbolic links leverage industry standard protocols
  • If implemented correct it becomes easier to migrate as symbolic links can be moved all around without impact. 
  • Easy to recreate a symbolic links

You could even consider putting in 3DFS that allows file updates, in that case it becomes secondary storage with backup integrated in its veins.
 

Userlevel 3
Badge +3

Hi @Onno van den Berg 

haven’t thought of that - but that is really brilliant. I never really worried about the driver - but it does have it’s disadvantages, especially on Windows where updates may need reboots etc. There was a time where Windows didn’t support symbolic links - and Linux archiving wasn’t really a big thing, so considering to implement this with Symlinks was probably never on the table when dev first envisioned the product. But time has moved on, and this really makes sense to look into it more. 

The drawback of 3DFS is that it needs a server hosting it which today is a Linux machine which may not always be desirable by a pure Windows customer for Windows file archiving. But it could be the alternative or an addition to the current “driverless archiving” which requires Explorer Add-In or Webconsole. 

Let’s see how this goes - but I do understand the benefits of this for customers that are willing to implement a 3DFS server. 

And when we get the chance to meet in TF, I am happy to have a dinner with you at Char steak house… we have to do this again :)

Userlevel 7
Badge +19

The challenge for customers who are less familiar running a Linux VM could be taken away if Commvault would provide it as an OVF template / appliance a-like. In this case, not a specific template for 3DFS, but a generic by Commvault hardened (based on industry standards) appliance like solution that get's is function based on the role it needs to fill in. It could be used as network gateway, MA and considering most core roles like web server, Command Center can be run on Linux, you could envision to role it out like this. In public cloud the scale-out architecture does a similar thing for VSA already and if I recall correctly this implementation will be used for more roles in the near future, so why not have it available in your own DC that is running on VMware. 

Char steak:-) Best restaurant in town :-) Count me in! 

Userlevel 4
Badge +12

@PraveenV 

Hi Praveen

With the new Risk Analysis SKU’s if we use a 10 VM pack can we analyse ‘all data sources’ within those VMs (eg File, SQL, Exchange)

Trying to understand if there is a need to use the TB risk analysis SKU

 

 

PS: (Just watching your last video with Patrick - Its very well put together - and now glad to see SDG and FSO combined as one) 

Reply