How to run online backup / Datastore GC / Tar Optimization using curl in CQ5.5

Use case: Curl command for online backup / Datastore GC / Tar Optimization for CQ5.4 does not work for CQ5.5 any more

Reason: Many CRX related operation is moved to JMX. Now you can use any JMX console or JMX API to invoke these operations.

Solution: You can use following curl commands


For online backup


curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=<PATH OF BACKUP>/test.zip

Following options are available for backup

Delay (Default is 10 mili second)

curl -u <UID>:<PASSWORD> -X POST
http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/a/BackupDelay?value=<TIME-IN-MILISECOND>


Note After running this from front end you will see (null) as response. You have to check CRX log to verify if backup started. In CRX log (/crx-quickstart/logs/error.log) you would see message like this

*INFO* [Backup Worker Thread] com.day.crx.core.backup.Backup Backup started.
*INFO* [Backup Worker Thread] com.day.crx.core.backup.Backup Read size (1.234 gb) in 1.97s


OR Use following CURL for online backup


curl -u admin:admin --data "delay=3&force=false&target=test3.zip" http://HOST:PORT/libs/granite/backup/content/admin/backups/

Where,
delay: Time delay between write block.
force: To override last one
Target: Absolute path of backup. If path is not provided and just file name is given, Then it will use current path of installation.

Once backup is started, You can cancel it using following curl command

curl -u admin:admin http://HOST:PORT/libs/granite/backup/content/admin/backups.cancel.html

You can also track online progress using following URL

http://HOST:PORT/libs/granite/backup/content/admin.html




For Datastore GC


curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/runDataStoreGarbageCollection/java.lang.Boolean

To delete data and delay as 2

curl -u admin:admin -X POST --data "delete=true&delay=2" http://HOST:PORT/system/console/jmx/com.adobe.granite%3Atype%3DRepository/op/runDataStoreGarbageCollection/java.lang.Boolean

CRX Log message to look for

*INFO* [127.0.0.1 [1332343886706] POST /system/console/jmx/com.adobe.granite%3Atype%3DRepository/op/runDataStoreGarbageCollection/java.lang.Boolean HTTP/1.1] com.day.crx.sling.server.impl.jmx.GarbageCollection Scanning /libs/wcm/core/content/siteadmin/actions/create/menu/createPage

Automate DataStore GC (AEM 5.6 onward)

Option 1: Using Configuration

create new configuration name 

<some path for config>/com.day.crx.sling.server.impl.jmx.GarbageCollectionConfig-DatastoreGarbageCollector.xml with following values

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jcr:primaryType="sling:OsgiConfig"
    jobName="Datastore Garbage Collector"
    schedule="00 00 23 ? * SUN"
    deleteNodes="{Boolean}true" />

This mean that run Datastore Garbage collection every Sunday at 11 PM.

Option 2: Using CURL and CRON job. Create a Cron job on your machine and run CURL command given above.

 
For Tar optimization

curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/startTarOptimization/

CRX Log Message to look for

com.day.crx.persistence.tar.file.TarFile /crx-quickstart/repository/workspaces/crx.default/data_00000.tar id:0 length:75436032 append:75435008 943288331 optimize: 2420224


Similarly you can use CURL command to different JMX operation listed in

http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository






How to identify master using Curl Command And then run backup only on master

ISMASTER=`curl -s -u admin:admin http://HOST:PORT/system/console/jmx/com.adobe.granite:type=Repository | grep -A 1 crx.cluster.master | sed 's/\<\/*td\>//g' | awk 'NR>1' | awk '{ print $1 }'`
if [ $ISMASTER != 'true' ]; then
echo "Not running backup, that is the slave node";
exit 1;
else
echo "Running online backup this is master Node"
# Your curl command for online backup goes here
fi;

Special Thanks to Andrew Khoury from Adobe for providing this CURL command.


Use firebug to get complete URL and then replace it in above curl command.