How to host Adobe Dynamic Tag Management System files in CQ dispatcher

Use Case: You often have situation where for marketing and analytics purpose you have to reply on dev team to push tracking code or tag management code or inclusion of any third party client side library. Satellite Search and Discovery provide great way to abstract client side tracking or tagging changes for marketing and analytics with development.
Documentation on Tag Management can be found here

Challenges: 

1) Update to CQ could happen out side dev cycle. For this make sure that satellite changes are completely tested before using it in production.
2) Since Satellite is similar to SAS service, changes in satellite could also cause some module not to work as expected. For that you can keep track of changes in satellite side and test that module.
3) Some time satellite load is slow if it is loaded from there hosted service. For this you can host satellite code on dispatcher and use some script to update it every time there is any change.

Include satellite script to CQ:

You can simply use,

<script type="text/javascript" src="SOME-PATH.js" ></script>

you can also use run mode specific configuration to include dev or prod specific script to your side.

Set<String> runModes = sling.getService(SlingSettingsService.class).getRunModes();

if(!runModes.contains("author")) { 

   if(runModes.contains("prod"){

             //Include prod satellite code

   }else{

            //Include dev satellite code

   }

}

Host satellite Script on Dispatcher:

For performance you can host satellite on dispatcher itself and then include it in your file. For this satellite provide a feature for deploy Hook. Deploy hook URL is called every time there is a change in any configuration or any rules are published. 

If you want to host satellite files on dispatcher then you can give this deploy hook URL as your production server URL path that you want to call every time there is any change. For example I want call a servlet or any script on change and update change in dispatcher and all other publish and author instance.

Here I am using python script to make this update,  Process is like this,

• Changes made in satellite
• Satellite call a dispatcher URL
• Dispatcher URL invoke python script (You need rewrite rule to do that)
• Python script checks if this is staging or production server
• Based on that it get corresponding satellite files which is in zip fomat
• Script unzip file, remove existing files from dispatcher if present and put it in dispatcher in certain location
• Then it calls other dispatchers to update files as well
• Then it issue upload request to upload changed file to author
• After that it issue tree activation request to update these files on all publish. (This step is required in case some one clear dispatcher cache).
• In order to avoid infinite loop within all dispatchers, one dispatcher call other dispatcher with a URL param indicating not to call other dispatcher.
• for UID:PWD you can use non admin users that only have access to satellite files, make sure that they have activation rights as well.

You can use 

Note: I am using old version of python, You can reduce code with latest version.

#!/usr/bin/python

import urllib2

import shutil

import urlparse

import os

import sys

import zipfile

import cgitb; cgitb.enable()

import cgi

import socket

import urllib

#import pwd

#import grp

#Global Var

#Read URL from path

#Staging IP List contain list of IP for Stage

staging_IP_list = ["X.X.X.X","X.X.X.X"]

#Production IP list

production_IP_list = ["Y.Y.Y.Y","Y.Y.Y.Y","Y.Y.Y.Y","Y.Y.Y.Y"]

#This is required to avoid circular loop

form = cgi.FieldStorage()

checked = form.getvalue("checked")

addr = socket.gethostbyname(socket.gethostname())

#This folder path is required to avoid permission issue.

rootFolder = "../some/folder/in/dispatcher"

#Need to create rewrite mapping for this to work.

pingUrl = "/some/path/for/dispatcher?checked=true"

staging_satellite_url="THIS IS THE URL WHERE DEV SATELLITE FILE IS HOSTED"

production_satellite_url="THIS IS THE URL WHERE PROD SATELLITE FILE IS HOSTED"

url=staging_satellite_url

author_content_folder="FOLDER NAME WHERE YOU WANT THIS FILE TO GO"

author_content_subfolder="SUBFOLDER NAME GIVEN BY SATELLITE /"

satellite_js_file_name="SATELLITE FILE NAME GIVEN BY SATELLITE"

staging_author_server="YOUR-STAGING-AUTHOR-SERVER"

production_author_server="YOUR-PRODUCTION-AUHTOR-SERVER"

curl_ping_url=staging_author_server

script_file_list=[]

#Destination Path

print "Content-type: text/html; charset=iso-8859-1\n\n"

print '''<HTML>'''

print '''<TITLE>Satellite Ping Check</TITLE><body>'''

#print '''<br>url I got host as''',addr

#This method override url open to make just head request

class HeadRequest(urllib2.Request):

def get_method(self):

return "HEAD"

#Method to ping URL to another server

def pingURL(customURL):

try:

response = urllib2.urlopen(HeadRequest(customURL))

except:

print '''<br>We failed to reach a server.'''

#Method that will ping other server based on IP address

def pingOtherServer():

for eachIp in staging_IP_list:

if eachIp==addr:

for eachIp2 in staging_IP_list:

if eachIp2!=addr:

resp = pingURL("http://"+eachIp2+pingUrl)

break

for eachIp in production_IP_list:

if eachIp==addr:

for eachIp2 in production_IP_list:

if eachIp2!=addr:

resp = pingURL("http://"+eachIp2+pingUrl)

break

#This is required to keep those files to author

def pingauthorServer():

filepath = rootFolder+"/"+author_content_subfolder+satellite_js_file_name

#Curl command to upload satellite file

os.system('curl -u UID:PWD -F@TypeHint="nt:file" -Ftype="file" --upload-file '+filepath+' '+curl_ping_url+author_content_folder+author_content_subfolder)

filepath=rootFolder+"/"+"selector.js"

#Curl command to upload selectors.js file

os.system('curl -u UID:PWD -F@TypeHint="nt:file" -Ftype="file" --upload-file '+filepath+' '+curl_ping_url+author_content_folder)

for script_file in script_file_list:

filepath=rootFolder+"/"+author_content_subfolder+"scripts/"+script_file

os.system('curl -u UID:PWD -F@TypeHint="nt:file" -Ftype="file" --upload-file '+filepath+' '+curl_ping_url+author_content_folder+author_content_subfolder+"scripts/")

#Curl command to activate files to publish instance

os.system('curl -u UID:PWD -Fcmd=activate -Fignoredeactivated=true -Fonlymodified=false -Fpath='+author_content_folder+' '+curl_ping_url+'/etc/replication/treeactivation.html')

#Method to delete existing folder before extracting new one

def deleteFileOrFolder(directory):

    if os.path.exists(directory):

        try:

            if os.path.isdir(directory):

                print '''<br>removing folder<b>''',directory

                shutil.rmtree(directory)

                print '''<br>Creating''',directory

                os.makedirs(directory)

            else:

                print '''<br>removing file<b>''',directory

                os.remove(directory)

        except:

            print '''<br>Ecxeption''',str(sys.exc_info())

    else:

        print '''<br>not found''',directory

        print '''<br>Creating''',directory

        os.makedirs(directory)

#Method to set satellite url based on IP address. If this is production server then set URL as production

def seturl():

for eachIp in production_IP_list:

if eachIp==addr:

global url

url=production_satellite_url

global satellite_js_file_name

satellite_js_file_name="YOUR-SATELLITE-FILE-NAME.js"

global curl_ping_url

curl_ping_url=production_author_server

break

def extract():

zip_file = zipfile.ZipFile(fileName, 'r')

#print '''file name is ''',fileName

for files in zip_file.namelist():

print '''<br>files in zip''',files

myfile_path=rootFolder+"/"+files

#print '''<br> Yogesh ''',myfile_path

if myfile_path.endswith("/"):

#print '''<br>I am in if and myfile_path is ''',myfile_path

if not os.path.exists(myfile_path):

os.makedirs(myfile_path)

else:

if files.find("/scripts/") != -1:

script_file_list.append(files.split('/')[-1])

#print '''<b> found script file with name <br>''',rootFolder+"/"+author_content_subfolder+"scripts/"+files.split('/')[-1]

#print '''<br>I am here and myfile_path is ''',myfile_path

data = zip_file.read(files)

myfile = open(myfile_path, "w+")

myfile.write(data)

myfile.close()

zip_file.close()

#Setting URL to production if this is production server. By default it is always staging server

seturl()

#print '''<br>url I got is''',url

fileName = url.split('/')[-1].split('#')[0].split('?')[0]

print '''<br>filename I got is''',fileName

#Delete all file and folder before creating them

#deleteFileOrFolder(rootFolder+"/"+fileName)

deleteFileOrFolder(rootFolder)

r = urllib2.urlopen(urllib2.Request(url))

try:

fileName = rootFolder+"/"+fileName

f=open(fileName, 'wb')

urllib.urlretrieve(url,fileName)

finally:

    r.close()

#zfile = zipfile.ZipFile(fileName)

extract()

#zfile.extractall(rootFolder)

#os.system('jar -xvf '+fileName)

#Do it only from one server

if checked is None:

pingOtherServer()

pingauthorServer()

print '''</body>'''

print '''</HTML>'''

Happy tagging and tracking. Let me know if you have any question.

AEM 6 provide this feature OOTB for that go to http://HOST:PORT/miscadmin#/etc/cloudservices/dynamictagmanagement and enter your DTM info

Note: Please note that there could be other tools that are capable of doing similar things. You can use similar approach there as well. This post has no mean to say that you should use satellite search and discovery for similar use case.