Difference between revisions of "IBM CMOD Cache Filesystems"
m (IBM CMOD Cache Optimization - link) |
m (Changed title. →Too late, I already deleted files from my CMOD cache filesystem. Help!) |
||
| (10 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
==What is the IBM CMOD Cache filesystem?== | ==What is the IBM CMOD Cache filesystem?== | ||
The | The OnDemand cache filesystem, often named “arscache” is a pool of storage on your server that Content Manager OnDemand uses to store documents for fast loading or retrieval. Application Groups are configured by default to load data into the cache filesystem, and migrate it to secondary storage at a later time. Migrating the data from the CMOD cache to secondary storage doesn’t delete it - the caches are kept at 80% by default, to help make retrievals for end users faster. One of the easiest ways to tune the OnDemand Cache performance on large Content Manager OnDemand installations is to increase this number to over 95%. | ||
== Where do I configure the behavior of the CMOD cache? == | == Where do I configure the behavior of the CMOD cache? == | ||
| Line 10: | Line 10: | ||
===Diagram: Configuring IBM CMOD Cache behavior=== | ===Diagram: Configuring IBM CMOD Cache behavior=== | ||
[[File:CacheConfiguration.png|720px]] | |||
==How does the cache get filled?== | ==How does the cache get filled?== | ||
As you load data into IBM Content Manager OnDemand, data is placed into the caches for fast access. The only exception to this rule is if you choose not to load Application Group data to the cache at all - which is generally reserved for situations where documents need to be archived for legal or regulatory reasons, but are rarely, if ever, accessed. | As you load data into IBM Content Manager OnDemand, data is placed into the caches for fast access by end users. The only exception to this rule is if you choose not to load Application Group data to the cache at all, and send them directly to secondary storage - which is generally reserved for situations where documents need to be archived for legal or regulatory reasons, but are rarely, if ever, accessed. | ||
If you have more than one cache filesytem, | If you have more than one cache filesytem, OnDemand chooses the cache with the most free space, and puts the compressed objects generated by the load there. | ||
==How does the cache get emptied?== | ==How does the cache get emptied?== | ||
You can reduce the amount of data in the cache filesystems with the [[arsmaint]] command. You will want to migrate any data with the <code> arsmaint -m </code> command first, then the <code>arsmaint -c</code> command. | You can reduce the amount of data in the cache filesystems with the [[arsmaint]] command. You will want to migrate any data with the <code>arsmaint -m </code> command first, then run the <code>arsmaint -c</code> command to delete eligible items from the cache. You can combine both options in the same command <code>arsmaint -mc </code> to perform both tasks. While it's possible to run both cache migration and expiration at the same time in two different processes, it's not recommended. | ||
== What is cache expiration? == | == What is cache expiration? == | ||
CMOD Cache expiration is the process by which OnDemand reduces the amount of data inside the caches, in order to make room for new data that is loaded into your Content Manager OnDemand server. It does this by removing data that has already been migrated to secondary storage | CMOD Cache expiration is the process by which OnDemand reduces the amount of data inside the caches, in order to make room for new data that is loaded into your Content Manager OnDemand server. It does this by removing data that has already been migrated to secondary storage or is otherwise eligible for expiration - starting with the oldest documents first - until the filesystems are approximately 80% full by default. As mentioned above, on large IBM CMOD servers that have multiple terabytes of cache, adjusting the minimum and maximum parameters makes optimal use of the expensive storage that the CMOD cache is built with. The only reason to maintain an excessive amount of free space in your cache is to ensure there is room to load incoming files if data can't be written to secondary storage for some reason. You can visit the [[arsmaint]] page to find out more about how you can create free space in the IBM CMOD cache filesystem. | ||
==What is cache optimization?== | ==What is cache optimization?== | ||
Fast and reliable storage is expensive and finite. In order to make the most out of the space you have in your IBM CMOD Cache filesystems, you must store more of the data that users are actually accessing, but only for the period of time that they are actually accessing them. It's possible to optimize the cache contents so that you’re using precisely the amount of disk necessary to quickly perform the required retrievals, but still have enough space to load new data. An optimized cache contains only the data people are accessing. Unfortunately, there's no built-in tool for performing the Content Manager OnDemand Cache, but [https://cmod.cloud/ibm-cmod-cache-optimization/ CMOD Cache optimization] is offered as a service by companies that specialize in [https://TenaciousConsulting.com IBM CMOD Professional Services]. | |||
==How can I optimize the IBM CMOD Cache?== | ==How can I optimize the IBM CMOD Cache?== | ||
If your IBM CMOD Cache Filesystems are large ( | If your IBM CMOD Cache Filesystems are large (for example, 1 terabyte / 1000 gigabytes) then the default setting for the <code>arsmaint -c </code> will leave 20% - 200 gigabytes of empty cache. If you have multiple 1TB cache filesytems, and you only load 20GB per day, you could have a huge amount of wasted space in your caches. You can adjust this number with the <code>-n </code> and <code>-x </code> options for [[arsmaint]] to set new maximum and minimum caches sizes during cache expiration. | ||
The other ways to tune the cache filesystems are to | The other ways to tune the cache filesystems are to analyze the patterns of retrievals of Application Group data, and change the cache retention to match actual end user usage, and to use data from your load and retrieval patterns to determine your optimal cache size. | ||
== Can I delete files in the cache manually? == | == Can I delete files in the cache manually? == | ||
NO! The cache is only to be managed | NO! The cache is only to be managed with the [[arsmaint]] utility! The structure of the OnDemand Cache Filesystem is complex, and deleting files or directories inside the Content Manager OnDemand cache could lead to documents becoming unavailable to end users, and potentially irrecoverable data loss. You should be running cache expiration regularly. | ||
== | == How does CMOD Cache Expiration work? == | ||
If you have a backup of your cache data in your enterprise backup system, you can probably recover the missing data. | The simple answer is that CMOD does the best it can to remove data that no longer meets the retention requirements specified in the Application Group configuration to reduce the amount of data in the cache to the goal - which is 80% full (with 20% free) by default, but it can be modified with the <code> -n</code> and <code> -x</code> parameters to the [[arsmaint]] command. | ||
== I deleted files from my CMOD Cache filesystem. Help!== | |||
If you have a backup of your cache data in your enterprise backup system, you can probably recover the missing data. Contact a company that provides [https://cmod.cloud/professionalservices/ Professional Services for CMOD] for assistance in recovering data. They have tools to help you restore and verify the contents of your cache, or to pull data from secondary storage back into the CMOD cache. | |||
== What are the retr and migr directories? == | == What are the retr and migr directories? == | ||
| Line 40: | Line 44: | ||
== How do I add more cache filesystems? == | == How do I add more cache filesystems? == | ||
Ask your System Administrator or Storage Management team to add an additional cache filesystem to your server (create /arscache/cache03... cache04, etc.) and simply add that filesystem as an entry in the CMOD cache configuration file, normally ars.cache. If you have more than one Content Manager OnDemand instance configured on your IBM CMOD server, then you’ll need to edit the file specified by SRVR_SM_CONFIG in the instance definition in your [[ars.ini]] configuration file. But if you already have multiple OnDemand Caches, then you can simply choose to expand the filesystems you already have. | |||
== Which cache do I expand? == | |||
If you have multiple CMOD Cache Filesystems, and you're running low on space, you can simply add more storage to ONE of your caches, instead of adding space to ''each'' IBM Content Manager OnDemand cache filesystem. OnDemand allocates data to the cache filesystem with the largest number of free 'blocks' when loading data. | |||
== Can I remove a cache filesystem? == | == Can I remove a cache filesystem? == | ||
| Line 46: | Line 53: | ||
== Can I balance the data in the cache filesystem? == | == Can I balance the data in the cache filesystem? == | ||
Content Manager OnDemand automatically balances the amount of data in caches by placing new CMOD object files in the cache with the most free space, measured in blocks of free space in the filesystem. If you have a large number of cache filesystems with dramatically different utilization, | Content Manager OnDemand automatically balances the amount of data in caches by placing new CMOD object files in the cache with the most free space, measured in blocks of free space in the filesystem. If you have a large number of cache filesystems with dramatically different utilization, you probably have a configuration issue that needs to be corrected. | ||
== Related Articles == | == Related Articles == | ||
Latest revision as of 17:36, 27 October 2021
IBM Content Manager OnDemand Cache Filesystem
Here are some frequently asked questions about the IBM CMOD cache.
What is the IBM CMOD Cache filesystem?
The OnDemand cache filesystem, often named “arscache” is a pool of storage on your server that Content Manager OnDemand uses to store documents for fast loading or retrieval. Application Groups are configured by default to load data into the cache filesystem, and migrate it to secondary storage at a later time. Migrating the data from the CMOD cache to secondary storage doesn’t delete it - the caches are kept at 80% by default, to help make retrievals for end users faster. One of the easiest ways to tune the OnDemand Cache performance on large Content Manager OnDemand installations is to increase this number to over 95%.
Where do I configure the behavior of the CMOD cache?
Inside the Content Manager OnDemand Administrator Client, in the Application Group window, on the Storage Management tab, there’s a button labelled ‘Advanced’ that will allow you to change how Application Group data is managed.
Diagram: Configuring IBM CMOD Cache behavior
How does the cache get filled?
As you load data into IBM Content Manager OnDemand, data is placed into the caches for fast access by end users. The only exception to this rule is if you choose not to load Application Group data to the cache at all, and send them directly to secondary storage - which is generally reserved for situations where documents need to be archived for legal or regulatory reasons, but are rarely, if ever, accessed.
If you have more than one cache filesytem, OnDemand chooses the cache with the most free space, and puts the compressed objects generated by the load there.
How does the cache get emptied?
You can reduce the amount of data in the cache filesystems with the arsmaint command. You will want to migrate any data with the arsmaint -m command first, then run the arsmaint -c command to delete eligible items from the cache. You can combine both options in the same command arsmaint -mc to perform both tasks. While it's possible to run both cache migration and expiration at the same time in two different processes, it's not recommended.
What is cache expiration?
CMOD Cache expiration is the process by which OnDemand reduces the amount of data inside the caches, in order to make room for new data that is loaded into your Content Manager OnDemand server. It does this by removing data that has already been migrated to secondary storage or is otherwise eligible for expiration - starting with the oldest documents first - until the filesystems are approximately 80% full by default. As mentioned above, on large IBM CMOD servers that have multiple terabytes of cache, adjusting the minimum and maximum parameters makes optimal use of the expensive storage that the CMOD cache is built with. The only reason to maintain an excessive amount of free space in your cache is to ensure there is room to load incoming files if data can't be written to secondary storage for some reason. You can visit the arsmaint page to find out more about how you can create free space in the IBM CMOD cache filesystem.
What is cache optimization?
Fast and reliable storage is expensive and finite. In order to make the most out of the space you have in your IBM CMOD Cache filesystems, you must store more of the data that users are actually accessing, but only for the period of time that they are actually accessing them. It's possible to optimize the cache contents so that you’re using precisely the amount of disk necessary to quickly perform the required retrievals, but still have enough space to load new data. An optimized cache contains only the data people are accessing. Unfortunately, there's no built-in tool for performing the Content Manager OnDemand Cache, but CMOD Cache optimization is offered as a service by companies that specialize in IBM CMOD Professional Services.
How can I optimize the IBM CMOD Cache?
If your IBM CMOD Cache Filesystems are large (for example, 1 terabyte / 1000 gigabytes) then the default setting for the arsmaint -c will leave 20% - 200 gigabytes of empty cache. If you have multiple 1TB cache filesytems, and you only load 20GB per day, you could have a huge amount of wasted space in your caches. You can adjust this number with the -n and -x options for arsmaint to set new maximum and minimum caches sizes during cache expiration.
The other ways to tune the cache filesystems are to analyze the patterns of retrievals of Application Group data, and change the cache retention to match actual end user usage, and to use data from your load and retrieval patterns to determine your optimal cache size.
Can I delete files in the cache manually?
NO! The cache is only to be managed with the arsmaint utility! The structure of the OnDemand Cache Filesystem is complex, and deleting files or directories inside the Content Manager OnDemand cache could lead to documents becoming unavailable to end users, and potentially irrecoverable data loss. You should be running cache expiration regularly.
How does CMOD Cache Expiration work?
The simple answer is that CMOD does the best it can to remove data that no longer meets the retention requirements specified in the Application Group configuration to reduce the amount of data in the cache to the goal - which is 80% full (with 20% free) by default, but it can be modified with the -n and -x parameters to the arsmaint command.
I deleted files from my CMOD Cache filesystem. Help!
If you have a backup of your cache data in your enterprise backup system, you can probably recover the missing data. Contact a company that provides Professional Services for CMOD for assistance in recovering data. They have tools to help you restore and verify the contents of your cache, or to pull data from secondary storage back into the CMOD cache.
What are the retr and migr directories?
Someone’s been spelunking! The retr and migr directories inside the cache help CMOD retrieve and migrate documents. There’s a very good presentation on the OnDemand User’s Group forums on the cache filesystem internals. Registration on the forums is required to access the presentation - and new registrations are normally approved within 48 hours.
How do I add more cache filesystems?
Ask your System Administrator or Storage Management team to add an additional cache filesystem to your server (create /arscache/cache03... cache04, etc.) and simply add that filesystem as an entry in the CMOD cache configuration file, normally ars.cache. If you have more than one Content Manager OnDemand instance configured on your IBM CMOD server, then you’ll need to edit the file specified by SRVR_SM_CONFIG in the instance definition in your ars.ini configuration file. But if you already have multiple OnDemand Caches, then you can simply choose to expand the filesystems you already have.
Which cache do I expand?
If you have multiple CMOD Cache Filesystems, and you're running low on space, you can simply add more storage to ONE of your caches, instead of adding space to each IBM Content Manager OnDemand cache filesystem. OnDemand allocates data to the cache filesystem with the largest number of free 'blocks' when loading data.
Can I remove a cache filesystem?
Yes, and no. You can remove a filesystem from the ars.cache configuration file, and CMOD will stop adding data to it during loads, and will expire data from it during cache expiration processing. However, your IBM CMOD server will NOT redistribute objects to other caches. If you want to consolidate your existing caches into a smaller collection of caches, visit Cache Consolidation for assistance.
Can I balance the data in the cache filesystem?
Content Manager OnDemand automatically balances the amount of data in caches by placing new CMOD object files in the cache with the most free space, measured in blocks of free space in the filesystem. If you have a large number of cache filesystems with dramatically different utilization, you probably have a configuration issue that needs to be corrected.
Related Articles
ARS0199E Unexpected empty directory in cache
ARS0198E Invalid ownership and/or permissions on cache file/directory
ARS0111E OnDemand is unable to determine the link for file
In what situations will IBM CMOD expire documents form the cache?