Use Case: Something went wrong during Asset upload and not all steps in Update Asset workflow get executed successfully.
Impact: Some time when Assets are not uploaded properly or something went wrong during upload process, not all steps in update asset workflow completes, as a result of which some time metadata or rendition is missing. You can probably run those script again to complete process but easiest way would be to put those asset back to workflow.
Solution:
You can use following script to do this
# !/bin/bash
# Author: Yogesh Upadhyay
# The host and port of the source server
SOURCE="HOST:PORT"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:ADMIN PASSWORD"
#Filter path
ROOT_PATH="PATH OF ASSET"
IFS=$'\n'
#If you want to run for only those asset for which property is missing
MISSING_PROPERTY=dc:format
#Query to get all Dam Asset
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&property=jcr:content/metadata/@$MISSING_PROPERTY&property.operation=not&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
#echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED -d "model=/etc/workflow/models/dam/update_asset/jcr:content/model&payloadType=JCR_PATH&payload=$SINGLE_PATH" $SOURCE/etc/workflow/instances
sleep 3
done
Script to change wrong mimeType and dc:format of pdf document
# !/bin/bash
# Author: upadhyay.yogesh@gmail.com
# The host and port of the source server
SOURCE="localhost:4504"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:<PASSWORD>"
#Filter path
ROOT_PATH="<ROOT PATH>"
IFS=$'\n'
#Query to get all Dam Asset with extension pdf but with default mime type
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&nodename=*.pdf&property=jcr:content/renditions/original/jcr:content/jcr:mimeType&property.value=application/octet-stream&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED "-Fjcr:mimeType=application/pdf" $SOURCE$SINGLE_PATH/jcr:content/renditions/original/jcr:content
curl -s -u $SOURCE_CRED "-Fdc:format=application/pdf" $SOURCE$SINGLE_PATH/jcr:content/metadata
sleep 2
done
You can actually check for not equals property as well, But that will require custom predicate evaluator for not.
Some more interesting script:
Change thumbnail process arg to add additional thumbnail
SOURCE="localhost:4502"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:admin"
IFS=$'\n'
#Keep previous one as is this is not append operation,
#Off course this could be empty if you dont want any thumbnail
NEW_ARGS="[140:100],[48:48],[319:319],[210,210]"
PROCESS_PATH=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=/etc/workflow/models/dam/update_asset/jcr:content/flow&type=nt:unstructured&property=PROCESS&property.value=com.day.cq.dam.core.process.CreateThumbnailProcess&p.limit=-1" | tr "[" "\n" | sed 's/},{/\n/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$PROCESS_PATH"
curl -s -u $SOURCE_CRED "-FPROCESS_ARGS=$NEW_ARGS" $SOURCE$PROCESS_PATH
Remove all thumbnail and just keep original file
# !/bin/bash
# Author: upadhyay.yogesh@gmail.com
# The host and port of the source server
SOURCE="localhost:4502"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:admin"
IFS=$'\n'
#Filter path
ROOT_PATH="/content/dam<Your path>"
ALL_PATHS=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&nodename=*thumbnail*png&type=nt:file&p.limit=-1" | tr "[" "\n" | sed 's/},{/\n/g' | sed 's/ /%20/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
echo "Removing $SINGLE_PATH"
curl -u $SOURCE_CRED -F:operation=delete $SOURCE$SINGLE_PATH
done
# !/bin/bash
# Author: Yogesh Upadhyay
# The host and port of the source server
SOURCE="YOUR SERVER NAME"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:password"
#Filter path
ROOT_PATH="/content/dam"
DATE_FROM="2016-03-10"
DATE_TO="2016-03-16"
IFS=$'\n'
#If you want to run for only those asset for which property is missing
#Query to get all Dam Asset
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&daterange.property=jcr:created&daterange.lowerBound=$DATE_FROM&daterange.upperBound=$DATE_TO&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
#echo "\n"
echo "$ALL_PATHS"
#echo "\n"
for SINGLE_PATH in $ALL_PATHS
do
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED -d "model=/etc/workflow/models/dam/update_asset/jcr:content/model&payloadType=JCR_PATH&payload=$SINGLE_PATH" $SOURCE/etc/workflow/instances
sleep 3
done
Please check http://dev.day.com/docs/en/cq/current/workflows/wf-extending.html and http://dev.day.com/docs/en/cq/current/dam/customizing_and_extendingcq5dam/query_builder.html in case you want to achieve this through code.
Note: If you have some custom property in your Asset this might override those property. Please make sure that you have tested this script before using it.
Impact: Some time when Assets are not uploaded properly or something went wrong during upload process, not all steps in update asset workflow completes, as a result of which some time metadata or rendition is missing. You can probably run those script again to complete process but easiest way would be to put those asset back to workflow.
Solution:
You can use following script to do this
# !/bin/bash
# Author: Yogesh Upadhyay
# The host and port of the source server
SOURCE="HOST:PORT"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:ADMIN PASSWORD"
#Filter path
ROOT_PATH="PATH OF ASSET"
IFS=$'\n'
#If you want to run for only those asset for which property is missing
MISSING_PROPERTY=dc:format
#Query to get all Dam Asset
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&property=jcr:content/metadata/@$MISSING_PROPERTY&property.operation=not&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
#echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED -d "model=/etc/workflow/models/dam/update_asset/jcr:content/model&payloadType=JCR_PATH&payload=$SINGLE_PATH" $SOURCE/etc/workflow/instances
sleep 3
done
Script to change wrong mimeType and dc:format of pdf document
# !/bin/bash
# Author: upadhyay.yogesh@gmail.com
# The host and port of the source server
SOURCE="localhost:4504"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:<PASSWORD>"
#Filter path
ROOT_PATH="<ROOT PATH>"
IFS=$'\n'
#Query to get all Dam Asset with extension pdf but with default mime type
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&nodename=*.pdf&property=jcr:content/renditions/original/jcr:content/jcr:mimeType&property.value=application/octet-stream&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED "-Fjcr:mimeType=application/pdf" $SOURCE$SINGLE_PATH/jcr:content/renditions/original/jcr:content
curl -s -u $SOURCE_CRED "-Fdc:format=application/pdf" $SOURCE$SINGLE_PATH/jcr:content/metadata
sleep 2
done
You can actually check for not equals property as well, But that will require custom predicate evaluator for not.
Some more interesting script:
Change thumbnail process arg to add additional thumbnail
SOURCE="localhost:4502"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:admin"
IFS=$'\n'
#Keep previous one as is this is not append operation,
#Off course this could be empty if you dont want any thumbnail
NEW_ARGS="[140:100],[48:48],[319:319],[210,210]"
PROCESS_PATH=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=/etc/workflow/models/dam/update_asset/jcr:content/flow&type=nt:unstructured&property=PROCESS&property.value=com.day.cq.dam.core.process.CreateThumbnailProcess&p.limit=-1" | tr "[" "\n" | sed 's/},{/\n/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$PROCESS_PATH"
curl -s -u $SOURCE_CRED "-FPROCESS_ARGS=$NEW_ARGS" $SOURCE$PROCESS_PATH
Remove all thumbnail and just keep original file
# !/bin/bash
# Author: upadhyay.yogesh@gmail.com
# The host and port of the source server
SOURCE="localhost:4502"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:admin"
IFS=$'\n'
#Filter path
ROOT_PATH="/content/dam<Your path>"
ALL_PATHS=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&nodename=*thumbnail*png&type=nt:file&p.limit=-1" | tr "[" "\n" | sed 's/},{/\n/g' | sed 's/ /%20/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
echo "Removing $SINGLE_PATH"
curl -u $SOURCE_CRED -F:operation=delete $SOURCE$SINGLE_PATH
done
Put dam asset in workflow which is created between certain date
# !/bin/bash
# Author: Yogesh Upadhyay
# The host and port of the source server
SOURCE="YOUR SERVER NAME"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:password"
#Filter path
ROOT_PATH="/content/dam"
DATE_FROM="2016-03-10"
DATE_TO="2016-03-16"
IFS=$'\n'
#If you want to run for only those asset for which property is missing
#Query to get all Dam Asset
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&daterange.property=jcr:created&daterange.lowerBound=$DATE_FROM&daterange.upperBound=$DATE_TO&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
#echo "\n"
echo "$ALL_PATHS"
#echo "\n"
for SINGLE_PATH in $ALL_PATHS
do
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED -d "model=/etc/workflow/models/dam/update_asset/jcr:content/model&payloadType=JCR_PATH&payload=$SINGLE_PATH" $SOURCE/etc/workflow/instances
sleep 3
done
Note: If you have some custom property in your Asset this might override those property. Please make sure that you have tested this script before using it.
To allow for structure names that has spaces in the i would suggest you add the following to the scripts:
ReplyDeleteIFS=$'\n'
Also the part tr ",[" "\n" would not work for filenames containing ",".
tr "[" "\n" | sed 's/},{/\n/g' would solve that
Great post btw
Thanks you very much for your valuable input Johan, An update has been made to the post.
DeleteNot sure if it's my browser, but a parenthesis should exist after the equal sign in ALL_PATHS=(. Another should exist at the end. Great post btw, works perfectly.
ReplyDeleteI have a slightly off topic query... Is there a way to find if an asset is under any (other) workflow processing at the moment
ReplyDelete/Regards
Kanwal