Did you knew, that you can use PowerShell to automatically download files from a website using the Invoke-WebRequest cmd-let?
Download Files from Website using PowerShell
If you want to download multiple files form a website, you can automate this task using PowerShell. Maybe you need to download a lot of files from a website or maybe you need to download files that get updated regular and want to automate the task on a schedule. The possibilities are many and PowerShell can you help you with just that, using the Invoke-webRequest cmd-let.
The key component to automate download from one or more website, is the cmd-let: Invoke-Webrequest. With this command you can create all kind of automation script, to do the boring work for you. How about that? 🙂
How it can be used
Below is a small script that download all exe files from my download section on this blog. It saves the file in a Temp location on your local drive %Temp%\Download. At first, it crawls the URL for all download links with an .exe file extension. You will be presented for a list of files found before asked to continue the download. Once you press a key the download will start downloading all files from the list to the download location. This is just for you to get an idea of how it could be done. You can change the script to just the website or just the file extension, you need to download in your case.
Script Example with Invoke-Webrequest
# Set Download location
$DestinationFolder = $env:TEMP + “\Download”
# Define the URL of the download site
$URL = “https://www.techthatworks.net/downloads”
# Set ProgressPreference to SilentlyContinue to avoid download status messages
$ProgressPreference = ‘SilentlyContinue’
# Fetch the HTML content of the page
$Response = Invoke-WebRequest -Uri $URL -UseBasicParsing
# Extract all links ending with .exe
$DownloadLinks = ($Response.Links | Where-Object { $_.href -Match “\.exe$” }).href
# Remove duplicate links and sort
$UniqueLinks = $DownloadLinks | Select-Object -Unique
# Folder to store Files, create it if it doesn’t exist
if (-not (Test-Path -LiteralPath $DestinationFolder)) {
try {
New-Item -ItemType Directory -Path $destinationFolder -Force -ErrorAction Stop | Out-Null
Write-Host (“Created specified destination folder {0}” -f $DestinationFolder) -ForegroundColor Green
}
catch {
Write-Warning (“Could create destination folder {0}, check permissions. Exiting…” -f $DestinationFolder)
}
}
$UniqueLinks
$UniqueLinks.count
Read-Host -Prompt “Do you want to continue with download?”
# Download all .exe files and save them to specified destination folder
foreach ($url in $UniqueLinks) {
$FileName = Split-Path -Path $URL -Leaf
$DestinationPath = Join-Path -Path $destinationFolder -ChildPath $FileName
try {
Invoke-WebRequest -Uri $URL -OutFile $destinationPath -ErrorAction Stop
Write-Host (“Downloading {0} to {1}” -f $FileName, $DestinationPath) -ForegroundColor Green
}
catch {
Write-Warning (“Error downloading/saving {0} to {1}” -f $FileName, $DestinationFolder)
}
}
Conclusion on Invoke-WebRequest
This little script can be handy if you need to download a lot of files on a regular basis. The example is downloading “.exe” files, but you can change it to whatever filetype you want to download from whatever website you want do download from. Just remember to remove the line with Read-Host -Prompt. Otherwise, the script will not continue and start automatically, but while you debug it might be a nice step.
Feel free to drop me a comment or ask me a question about my script or the Invoke-Webrequest in general.
