Combine all log files in to 1 single log file?

Hey,
I’m potentially being stupid here, but I’ve tried looking around to see if there is a way to combine all log files produced by a PSADT App install into one single log file.
For example, I have an application that has a number of MSI’s that need to be installed in a particular order. PSADT creates it’s log file, and then each individual MSI generates a log file also - I would like all these log files to be combined in to 1 single log file.

I know PSADT has the compress in to a zip file option but I was hoping I could just combine all the logs in to 1 rather than use a zip file housing multiple log files.

The only trace I could see of this being mentioned was by @That-Annoying-Guy in a post but I couldn’t see anything in regards to how it was implemented so hoping he may be able to come to my rescue again :sweat_smile:

You could use Get-Content and Set-Content to join the log files eg.

Get-content -Path App1.log, App2.log, App3.log -Raw | 
    Set-Content -Path SingleLogFile.log -NoNewline

That’s one way but It needs to work regardless of the names of the log file.

I can help you get it going but I just don’t have much time like I did a few months ago.
If all you need is the merging of log files, that’s somewhat simple.
But If you need log file rotation on top, that’s trickier.

Thanks for the responses both!
To make sure I fully understand what you mean by Log file rotation, this is when if a log file started to get to a specific file size it would begin to break them into different parts? e.g. “Acrobat Reader DC 23.008.20458 V1(1).log”, “Acrobat Reader DC 23.008.20458 V1(2).log” and so on? If I’m correct then log file rotation isn’t something I need at this moment in time, it is just the merging of log files.

For instance, if I have an Acrobat Reader install using PSADT but it has a number of MSI’s/MSP’s I want all the MSI generated log files merged into the main log file for that PSADT instance. Then if I was to create a new version of Acrobat Reader using PSADT that will then have it’s own log file in the same manner, naturally these log files would have their own naming convention though - e.g. “Acrobat Reader DC 23.008.20458 V1.log” and then “Acrobat Reader DC 23.008.20458 V2.log”

I really appreciate your help however I most definitely do not want to be a burden on your already limited time, having the log files merge in the way I’ve mentioned is desirable for us but far from mandatory so I’m not stressing to implement this urgently.

Log rotation is to keep older versions of log files for the same package.
If you run the installation 3 times, you’ll get 3 log files.
I had A LOT of trouble getting this to work because the PSADT log file would pollute itself.

Below is the “merge log files to One log file” function you need.
It is based on PSADT’s New-ZipFile function so that the parameter names are identical.

#Region Merge-ToOneLogFile
Function Merge-ToOneLogFile {
<#
.SYNOPSIS
	Merges/Appends logs in $logTempFolder to a single file
.DESCRIPTION
	Creates a single new log file from folder of logfiles or an array of logfile paths
	Handles different log file encodings (UTF8, ANSI, etc)
	Output encoding is UTF8
	Merges log files in Chronological order using "Date Last Modified" timestamp
	Add header at top of each: ++ * LogFile: "<Name of log file>"
	Appends PSADT's log file as the last log file
	Appends the Exitcode from PSADT to LAST Line if possible
	Based on PSADT's New-ZipFile while matching parameters
	CAVEAT: Sub-folders of $SourceDirectoryPath are ignored
.PARAMETER DestinationArchiveDirectoryPath
	The path to the directory path where the merged log file will be saved.
.PARAMETER DestinationArchiveFileName
	The name of the merged log file
.PARAMETER SourceDirectoryPath
	The path to the directory to be archived, specified as absolute paths.
	CAVEAT: Use -SourceDirectoryPath or -SourceFilePath. Not both! -SourceDirectoryPath has precedence.
.PARAMETER SourceFilePath
	Array of file paths to the log file to be archived, specified as absolute paths.
	CAVEAT: Use -SourceDirectoryPath or -SourceFilePath. Not both! -SourceDirectoryPath has precedence.
.PARAMETER RemoveSourceAfterArchiving
	Remove the source path after successfully archiving the content. Default is: $false.
.PARAMETER OverWriteArchive
	Overwrite the destination archive path if it already exists. Default is: $false.
.PARAMETER ContinueOnError
	Continue if an error is encountered. Default: $true.
.PARAMETER ExitCode
	Used to insert the ExitCode of the script in the last line of the merged log file (as requested)
.EXAMPLE
	Merge-ToOneLogFile -DestinationArchiveDirectoryPath $configToolkitLogDir -DestinationArchiveFileName $DestinationArchiveFileName -SourceDirectory $logTempFolder -RemoveSourceAfterArchiving -ExitCode $ExitCode
.EXAMPLE
	Merge-ToOneLogFile -DestinationArchiveDirectoryPath $configToolkitLogDir -DestinationArchiveFileName "previouspkg_v1r1.log" -SourceFilePath $ArrayOfFilePaths
.NOTES
	We use this in 2 scenarios:
	-At the end of a package Run (when the script ends normally)
	-At the beginning of a package Run (to cleanup the log files from a *previous* package Run that ended AB-normally)
	CAVEAT: if one of the source log files has mixed encodings, this will cause 0x00 chars to get in
	TODO: We Need a *FAST* way to strip out these 0x00 chars without killing carriage returns
#>
	Param (
		[Parameter(Mandatory=$true,Position=0)]
		[ValidateNotNullorEmpty()]
		[string]$DestinationArchiveDirectoryPath,
		[Parameter(Mandatory=$true,Position=1)]
		[ValidateNotNullorEmpty()]
		[string]$DestinationArchiveFileName,
		[Parameter(Mandatory=$true,Position=2,ParameterSetName='CreateFromDirectory')]
		[ValidateScript({ Test-Path -LiteralPath $_ -PathType 'Container' })]
		[string[]]$SourceDirectoryPath,
		[Parameter(Mandatory=$true,Position=2,ParameterSetName='CreateFromFile')]
		[ValidateScript({ Test-Path -LiteralPath $_ -PathType 'Leaf' })]
		[string[]]$SourceFilePath,
		[Parameter(Mandatory=$false,Position=3)]
		[ValidateNotNullorEmpty()]
		[switch]$RemoveSourceAfterArchiving = $false,
		[Parameter(Mandatory=$false,Position=4)]
		[ValidateNotNullorEmpty()]
		[switch]$OverWriteArchive = $false,
		[Parameter(Mandatory=$false,Position=5)]
		[ValidateNotNullorEmpty()]
		[boolean]$ContinueOnError = $true,
		[Parameter(Mandatory=$false)]
		[ValidateNotNullorEmpty()]
		[int32]$ExitCode
	)
	
	Begin {
		## Get the name of this function and write header
		[string]${CmdletName} = $PSCmdlet.MyInvocation.MyCommand.Name
		
		Write-FunctionHeaderOrFooter -CmdletName ${CmdletName} -CmdletBoundParameters $PSBoundParameters -Header
	}
	Process {
		Try {
			If ($SourceDirectoryPath -eq $configToolkitLogDir) {
				Write-Log "ERROR: You are about to Merge *ALL* log files in [$configToolkitLogDir] and they would then be deleted. Aborted to prevent all log files from being trashed." -Source ${CmdletName}
				Throw "Merging *ALL* log files in [$configToolkitLogDir] aborted to prevent all log files from being trashed."
			}
			If ( -not (Test-Path -LiteralPath $SourceDirectoryPath -PathType Container )) { 
				Write-host "Log folder [$SourceDirectoryPath] does NOT exist. Nothing to do." -Source ${CmdletName}
				return
			}
			## Get the full destination path where the archive will be stored
			[string]$DestinationPath = Join-Path -Path $DestinationArchiveDirectoryPath -ChildPath $DestinationArchiveFileName -ErrorAction 'Stop'
			Write-Log -Message "Create a Merged log file with the requested content at destination path [$DestinationPath]." -Source ${CmdletName}

			## If the destination archive already exists, delete it if the -OverWriteArchive option was selected
			If ( ($OverWriteArchive) -and ($(Try {Test-Path $DestinationPath.trim() } Catch { $false })) ) {
				Write-Log -Message "An archive at the destination path already exists, deleting file [$DestinationPath]." -Source ${CmdletName}
				$null = Remove-Item -LiteralPath $DestinationPath -Force -ErrorAction 'Stop'
#			} ElseIf ( $(Try {Test-Path $DestinationPath.trim() } Catch { $false }) ) { #This returns $false on $null or "", Returns $False on " "
#				Write-Log -Message "[$DestinationPath] already exists. Rotating Logfiles." -Source ${CmdletName}
#				Rotate-Logfiles -LogFileNameToRotate $DestinationPath -LogFileParentFolder $configToolkitLogDir
			} 

			## Create the archive file
			If ($PSCmdlet.ParameterSetName -eq 'CreateFromDirectory') {
				## Create the MERGED file from a source directory
				
				$OutPutFileEncoding = "UTF8"	#Output encoding is UTF8
				$Utf8NoBomEncoding	= New-Object System.Text.UTF8Encoding($False)
				$Utf8Encoding 		= New-Object System.Text.UTF8Encoding($true)
				
				#If ( $(Get-ChildItem $SourceDirectoryPath).Count -gt 0 ) { #Does not work in Win10/ps5+
				If ((test-path $SourceDirectoryPath) -and ( $(Get-ChildItem $SourceDirectoryPath).Count -gt 0 )) {
					[String]$ListOfLogs = Get-ChildItem $SourceDirectoryPath | Sort-Object -Property LastWriteTime | select LastWriteTime,length,name | ft -AutoSize | Out-string
					[String]$ListOfLogs = "The following files will be Merged to one .Log file:`r`n" + $ListOfLogs +"`r"
					$ListOfLogs | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
					
					#Here we append all the log files into this script's log file, one after another in Chronological order (Oldest first)
					ForEach ($file in (Get-ChildItem $SourceDirectoryPath | Sort-Object -Property LastWriteTime)) {
						Write-Log "Collecting log file [$($file.Name)] to [$DestinationPath]..." -Source ${CmdletName}
						"+*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*+" | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
						"+* LogFile: `"$($file.FullName)`"" | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
						"+*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*+" | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
						" " | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue # b/c `n didn't work
						Try {
							[System.Collections.IEnumerable]$ContentIEnum = Get-Content $file.FullName -ErrorAction Stop | Out-String
						} Catch {
							[string]$ErrorMessage = "$($_.Exception.Message) $($_.ScriptStackTrace) $($_.Exception.InnerException)"
							"ERROR: Unable to collect [$($file.FullName)]: [$ErrorMessage]" | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
							Start-sleep -Seconds 11						
							#$RemoveSourceAfterArchiving = $false
						}
				        if ( ( $ContentIEnum -ne $null ) -or ( $ContentIEnum.Length -lt 1) ) {
							[System.IO.File]::AppendAllText($DestinationPath, $ContentIEnum, $Utf8Encoding)				
						} else {
		#					Write-Host "[ $($file.name) is $($content.Length) bytes ]"
							[System.IO.File]::AppendAllText($DestinationPath, "[ $($file.name) is $($content.Length) bytes ]", $Utf8Encoding)
				        }
						"`r`nEND OF LOG: `"$($file.FullName)`" " | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
						" `r`n `r`n " | Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue #2+1 blank lines with a space
						Start-Sleep -Milliseconds 80 #just in case
					}
					
					
					
					If (($ExitCode -ne $null) -or ($ExitCode -ne "")) {
						#Use $exitcode param and make it the last line in the AppendedLogFile
						"EXITCODE=($ExitCode) - $(Get-Date -format "dd/MM/yyyy HH:mm:ss").50"| Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
					} else {
						"EXITCODE=( ???? ) - $(Get-Date -format "dd/MM/yyyy HH:mm:ss").50"| Out-File $DestinationPath -Append -Encoding $OutPutFileEncoding -ErrorAction SilentlyContinue
					}
				} Else {
					Write-host "Log folder [$SourceDirectoryPath] is EMPTY. Nothing to do." -Source ${CmdletName}
				}
				
				#  If option was selected, recursively delete the source directory after successfully archiving the contents
				If ($RemoveSourceAfterArchiving) {
					Try {
						If ( (Test-Path -LiteralPath $SourceDirectoryPath ) -and (Test-Path -LiteralPath $DestinationPath -ErrorAction SilentlyContinue ) ) { 
							Write-Log "Merged file found [$DestinationPath]" -Source ${CmdletName}
							Write-Log -Message "Deleting the source directory [$SourceDirectoryPath] as contents have been successfully merged." -Source ${CmdletName}
							Remove-Item -LiteralPath $SourceDirectoryPath -Recurse -Force -ErrorAction 'Stop' | Out-Null
						} Else {
							Write-Log "`$SourceDirectoryPath [$SourceDirectoryPath] or `$DestinationPath [$DestinationPath] do not exist." -Source ${CmdletName} -Severity 3
							Write-Log "An error occurred while appending the log files: $($_.Exception.Message)" -Source ${CmdletName}
							If (-not $ContinueOnError) {
								Throw "`$SourceDirectoryPath [$SourceDirectoryPath] or `$DestinationPath [$DestinationPath] do not exist."
							}
						}
					} Catch {
						[String]$ResolveErrorString = "$($_.Exception.Message) $($_.ScriptStackTrace) $($_.Exception.InnerException)"
						Write-Log -Message "Failed to recursively delete the source directory [$SourceDirectoryPath]. `r`n[$ResolveErrorString]" -Severity 2 -Source ${CmdletName}
					}
				}
			} 
			Else { 	#Handle  -SourceFilePath parameter
				## Create the archive file from a list of one or more files
				[IO.FileInfo[]]$SourceFilePath = [IO.FileInfo[]]$SourceFilePath
				
				Write-Log -Message "Calling ${CmdletName} again but pointing it to temporary folder [$MyTempFolder]" -Source ${CmdletName}
				$MyTempFolder = Join-path -Path $envTemp -ChildPath $([System.IO.Path]::GetRandomFileName())
				New-Folder -Path $MyTempFolder -ContinueOnError $false
				ForEach ($File in $SourceFilePath) { #  Copy Each file to a temporary folder
					If ( Test-Path -LiteralPath $($File.FullName) ) {
						Copy-File -Path	$File.FullName -Destination $MyTempFolder -ContinueOnError $false
					} else {
						Write-Log -Message "Skipping [$($File.FullName)]. Does not exist." -Severity 2 -Source ${CmdletName}
					}
				}
				
				# Call this function again but point it to the temporary folder
				Write-Log -Message "Calling ${CmdletName} again but pointing it to temporary folder [$MyTempFolder]" -Source ${CmdletName}
				Merge-ToOneLogFile -DestinationArchiveDirectoryPath $DestinationArchiveDirectoryPath -DestinationArchiveFileName $DestinationArchiveFileName -SourceDirectory $MyTempFolder -RemoveSourceAfterArchiving 
				Write-Log -Message "Success! Back to 1st instance with the -SourceFilePath parameter" -Source ${CmdletName}
			}
		} Catch {
			[String]$ResolveErrorString = "$($_.Exception.Message) $($_.ScriptStackTrace) $($_.Exception.InnerException)"
			Write-Log -Message "Failed to *Merge* the requested file(s). `r`n[$ResolveErrorString]" -Severity 3 -Source ${CmdletName}
			If (-not $ContinueOnError) {
				Throw "Failed to *Merge* the requested file(s): $($_.Exception.Message)"
			}
		}	

		Write-host "No issues during Log File Merging, deleting Temporary log file [$logTempFile]" -Source ${CmdletName}
		Start-Sleep -Seconds 3
		If ($DebugLogHandling) {
			Write-host "`$DebugLogHandling = $DebugLogHandling. [$logTempFile] not deleted" -Source ${CmdletName}
		} Else {
			Remove-Item -Path $logTempFile -Force -ErrorAction SilentlyContinue  #Comment this line to keep log handling logs
		}
	
	} 
	End {
		Write-FunctionHeaderOrFooter -CmdletName ${CmdletName} -Footer
	}
	
} #Merge-ToOneLogFile
#EndRegion Merge-ToOneLogFile

CAVEAT:
I think you are going to have a problem with the name of the resulting merged log file.
They will all be the same unless you figure out a way to change that.

Disclaimer: Use at your own risk!

This looks awesome man, many thanks for sharing it with me.
I’ll be doing a fair bit of playing around and testing with it before trying to use it in any live deployment so I can spend some time figuring out the naming convention of the resulting merged log file.

Thanks again @That-Annoying-Guy, exceptional help as always!

1 Like

I’m having a bit of trouble trying to incorporate this into my deployment.
My assumption(which may be incorrect) is that I add your Merge-ToOneLogFile function to my AppDeployToolkitExtensions.ps1 and then call it within my Deploy-Applications.ps1? Or should I be injecting this straight into the AppDeployToolkitMain.ps1?

It must be called from AppDeployToolkitMain.ps1 and replace New-ZipFile in the Exit-Script function.

Note: I have an If statement to have both but this is quicker for you.

I know it’s not a script or related to PSADT, but if you have access to SCCM and CMTrace, it can merge logs for you.

That reminds me, I only use Legacy style log files in PSADT.
IOW, in AppDeployToolkitConfig.xml I have:
<Toolkit_LogStyle>Legacy</Toolkit_LogStyle>

my Merge-ToOneLogFile function will just treat CMTrace files like regular Legacy log files.

For merging CMTrace files I use 1E’s logfileviewer. It’s a bit unstable but it will merge multiple CMTrace files so that the time stamps between all the CMTrace files are sorted in one dialog window.

@drmike , can SCCM’s CMTrace.exe merge CMTrace files to one CMTrace file via command line
or is it like 1E’s logfileviewer utility without the bugs?

Aha! Thanks @That-Annoying-Guy, that makes sense. Got it working now! :smiley:

Thanks @drmike, I know that CMTrace can merge them but our previous set up was using batch and we had all logs merging in to the one so it was much tidier in our Logs folder, however whilst looking in to this I’m beginning to wonder if it would be better to have each log separate rather than merged as we had before.

@That-Annoying-Guy, my understanding is you can only open multiple log files and it’ll merge them rather than CMTrace being able to export a merged log file.

The merge function works just like the New-ZipFile function:

  • uses the temporary logs folder trigged by the <Toolkit_CompressLogs>True</Toolkit_CompressLogs> in the XML
  • opens each of the files in the temporary logs folder, one by one
  • merges them in a new log file
  • the finale log filename is the same as the regular PSADT log file (I think*)
  • erases the temporary logs folder

Note:
*My PSADT is modified to have unique Deploy-Application.PS1 files for each application.
The finale log filename is the same as my unique <applicationName_Version>.PS1

It’s up to you if you see value in this or not.
For us, it ONE file to request from the user to send over.
We don’t have mess of log files where we wonder which MSI/EXE log file(S) goes with which PSADT log file.

CMtrace has no command-line wisdom. All it can do is open files (I checked to see). You have to use the UI and choose merge.