SQL and Deadlocks – What, where and when are they exactly?

Working on for a customer where we were having some issues with some deadlocks. We couldn’t quite get the details on the events for a subset of reasons. Foremost, because the SQL Ring buffer events was recycled within one and a half hour during peak loads.
So we created a powershell script, which was set to run as a scheduled task, taking all deadlock events in the ring buffer and pairing it with relevant queries, one hour backwards.

Basically this script, run as a scheduled task every hour, will output all your deadlocks, for the past hour. If your system is not generating that much Ring buffer events, you can throttle it down or even just run the SQL query once in order to collect this data. Its just important to note, that these Ring buffer events will be overwritten by the good grace of the SQL server.

# Get todays date in order to stamp the output file correctly.
$Date = (Get-date).ToString("yyyy-MM-dd")

# The selected SQL instance to connect to
$SQLInstance = '<Servername>\<Instance>'
$Outputfile= "<Local folder location>"+$date+"Deadlocks.log"

# SQL Query to be performed
$SQLQuery = $("
SELECT *
FROM
  (SELECT xevent.query('(event/data/value/deadlock)[1]') AS DeadlockGraph,
          CAST (xevent.value('(event/@timestamp)[1]', 'nvarchar(30)') AS DATETIME) AS FullDate
   FROM
     (SELECT xevent.query('.') AS XEvent
      FROM
        (SELECT Cast(target_data AS XML) AS TargetData
         FROM sys.dm_xe_session_targets st
         JOIN sys.dm_xe_sessions s ON s.address = st.event_session_address
         WHERE s.NAME = 'system_health'
           AND st.target_name = 'ring_buffer') AS DATA CROSS apply targetdata.nodes ('RingBufferTarget/event[@name=""xml_deadlock_report""]') AS XEventData (xevent)) 
              AS src
   ) 
   AS RESULT
WHERE RESULT.FullDate >= Dateadd(HOUR, -1, Getdate());
")


# Collect deadlock data every hour, using MaxCharLenght to get full output back. 
$OrgFormat=$FormatEnumerationLimit
$FormatEnumerationLimit=-1
$DeadlockDataFromSQL=Invoke-Sqlcmd -Query $SQLQuery -ServerInstance $SQLInstance -MaxCharLength 10000 -QueryTimeout 0

# Output exception data to file using stream.
$stream = New-Object -TypeName System.IO.StreamWriter $OutPutFile, $true
foreach($line in $deadlockdataFROMSQL) 
	{
      	$stream.WriteLine(($line.DeadlockGraph | Out-string))
	}
$stream.close()
$FormatEnumerationLimit=$OrgFormat

Clean Disabled users from User Profile Store – Manage non-existing AD users in powershell without error

Basically there was a mismatch between the users in Active Directory and the users in the User Profile store of the SharePoint 2013 farm, for reasons unknown….
Want to know more? Have a look at one of the major differences between running SharePoint User Profile Synchronization service and SharePoint AD Import. Spoiler, one of them doesn’t delete users….

Anyway, before correcting the above I needed to clean up the User Profile, no need to have profiles that should be deleted. Maybe not rocket science, buuut this script does contain a nice little feature that checks if the user exists in Active Directory. This can be done with Get-ADUser, however that commandlet will return and error if the user does not exist, which is kinda hard to work with. That is what the dsquery part is there for.

#MySite Host URL
$mySiteHostUrl = "<mysite URL>"

# Static variable section. 
# Loading necessary context and creating arrays for later use
[Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server")
$mysite = Get-SPSite $mysiteHostUrl
$context = [Microsoft.Office.Server.ServerContext]::GetContext($mysite)
$upm =  New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($context)
$AllSPUsers=[System.Collections.ArrayList]@() 
$AllSpUsers= $upm.GetEnumerator()
$DisabledSPUsers=[System.Collections.ArrayList]@() 
$DisplayNameMisMatchUsers=[System.Collections.ArrayList]@() 

# Compares the SharePoint User list with Disabled users in Active Directory.
foreach($SPuser in $AllSPUsers)
	{
    $ExistUser=$Null
    $ExistUser=dsquery user -samid $SPUser.AccountName.Split('\')[1]
    if($ExistUser -eq $Null) 
		{
        $DisabledSPUsers.Add($SPUser)
        write-host $SPUSer.AccountName "Added"
        }
	    else
        {
        #write-host $SPUser.AccountName.Split('\')[1] "exists!"
        }
	}
# Creating list of username for later reference, this can be commented out.
$DisabledSPUsers.AccountName | Out-File D:\Temp\DisabledUsersInSP.txt

# This is the section that removes each of the disabled users from the User Profile store.
# Comment out this part and just get a list of users.
Foreach($DUser in $DisabledSPUsers)
    {
    $UPM.RemoveUserProfile($DUser.AccountName);
    write-host $DUser.AccountName "Removed"
    }

# Compares the SharePoint User list with the actual user in Active Directory, and then compares Displayname. 
# This to check for reused loginname
# DO NOT run this unless you already removed Disabled users 🙂
 foreach($SPuser in $AllSPUsers)
	{
    $user=get-ADuser $SPUser.AccountName.Split('\')[1] -Properties *
    if($User.DisplayName -ne $SPuser.DisplayName) 
		{
        	$DisplayNameMisMatchUsers.Add($SPUser)
		write-host $SPUSer.AccountName "DisplayName Mismatch"
       		}
	    else
        	{
        	#write-host $SPUser.AccountName.Split('\')[1] "exists!"
        	}
	}
# Creating list of usernames to handle acording to internal processes.
$DisplayNameMisMatchUsers.AccountName | Out-File D:\DisplayNameMisMatchUsers.txt  

Microsoft PKI with client certificate auto enrollment design specification

Next in the collection of design specifications is a Microsoft PKI with added client certificate auto enrollment settings.
A customer asked me if I could implement this and seeing as I’ve done a fair bit of Microsoft PKIs before, it didn’t seem like too much of a hassle. Of course I wanted it to be documented and shared with all of you.

All in all, it was a little less than 8 hours before client certificates started rolling out, without anyone noticing.

Microsoft PKI with client certificate auto enrollment design specification (located on Google drive)

Onedrive for Business Design Specification

Having worked on this document for quite some time I felt the need to publish it just about now. I see alot of attention on the Onedrive for business client as it has matured greatly over the past year and many national and international companies are looking towards what this solution can offer. The below design outlines a lot of these considerations, including network latency, legislation, features and of course a large collection of valuable links when considering the implementation of Onedrive for Business.
I will update document as my work brings me further into this area. Until then I hope you will enjoy this piece of work and benefit as much from it as I have.

Onedrive for Business Design Specification (located on Google drive)

Microsoft Cloud solution – Security, Auditing, Traceability and how do they react to data breach

You might have an interest in Microsoft cloud hosting security, auditing and / or data protection, just like me. I’ve collected a few good links, that should get you started.

First of, a little on how they react if breaches will happen, because they will at some point in some way. Major or minor…
How Would Microsoft Respond to a Data Breach of the Azure services?

Secondly, this white paper examines how Microsoft investigates, manages, and responds to security incidents within Azure.
Microsoft Azure Security Response in the Cloud

So how do they protect their physical data centers as well as your data and which certifications do they comply to?
Security, Audits, and Certifications

A little on their politics for how they manage and regard your data.
With Microsoft, you are the owner of your customer data.

And lastly a more wide perspective article.
5 questions every executive should be asking their security team

EICAR Standard Anti-Virus Test File

Recently a customer was testing antivirus scanning software, both on trafic and on servers/clients. This had to be tested on several environments, including production and really didnt want to use a real infected file. The following showed up, didn’t know about it, chances are there are others that doesnt know about this.

“The EICAR Standard Anti-Virus Test File or EICAR test file is a computer file that was developed by the European Institute for Computer Antivirus Research (EICAR) and Computer Antivirus Research Organization (CARO), to test the response of computer antivirus (AV) programs. Instead of using real malware, which could do real damage, this test file allows people to test anti-virus software without having to use a real computer virus.

Anti-virus programmers set the EICAR string as a verified virus, similar to other identified signatures. A compliant virus scanner, when detecting the file, will respond in exactly the same manner as if it found a harmful virus. Not all virus scanners are compliant, and may not detect the file even when they are correctly configured.

The use of the EICAR test string can be more versatile than straightforward detection: a file containing the EICAR test string can be compressed or archived, and then the antivirus software can be run to see whether it can detect the test string in the compressed file.”

EICAR test file – Wikipedia, the free encyclopedia

Steps to use it:
Create a .txt file on your drive, open your AV scanner software and create an exclusion on this file and location. Update the contents of the file with the referenced. Scanner software will not quarentine it with that name and location, however anywhere you move it, it should be detected and removed.

Configure Azure Application Proxy application for CRM Internet Facing Deployment

The task was to configure an existing CRM IFD, with an existing ADFS / Azure Application Proxy infrastructure.

CRM IFD deployment was already working and implemented for internal access. What was missing was the external access through Azure Application Proxy.
Following this guide: https://azure.microsoft.com/en-us/documentation/articles/active-directory-application-proxy-publish/
With the following configuration will get you there:

Note that once you have the CRM Internet facing deployment done, no changes are required on CRM or ADFS. The below is only the Azure Application Proxy configuration required for CRM internet facing deployment.

Logon to manage.windowsazure.com and create a new Application under Active Directory.
Important configuration is:
– External URL: You can use the same URL as internally, however make sure that CNAME record is created as well as certificate is uploaded. This is referred to as custom domain.
– Preauthentication Method: Passthrough
– Translate URL in headers: No

You need to add required CRM IFD service url’s:
organisation, authentication and/or discovery service as seperate applications.

References
https://azure.microsoft.com/en-us/documentation/articles/active-directory-application-proxy-publish/
https://azure.microsoft.com/en-us/documentation/articles/active-directory-application-proxy-custom-domains/
https://azure.microsoft.com/en-us/documentation/articles/active-directory-application-proxy-claims-aware-apps/

SharePoint Online – Performance – Basic Troubleshooting

Classic case: Customer reports in “SharePoint Online performance is slow. [period]”
Account Manager, Product Manager, Project Manager comes running 4 weeks after due date, customer wont accept the solution with the given performance. Ok…. Lets gather the tools for first steps:

Toolbox includes at this point
– PSping utility from PSTools(PSTools )
– Tracert utility from Microsoft (TraceRT )
– Browser Development Tools (F12 for Internet Explorer, Firefox or Chrome)

First step
Compare performance from your location with the client location using the Browser development tools. Are they the same?
If yes, problem usually lies in the configuration of SharePoint Online. Start testing from your location and don’t bother asking customer for client computer, remote login, network information.
If no, problem usually lies from customer client machine and SharePoint Online server. Start testing from customer client machine at their location. Ask them for information regarding their network configuration. Do they have proxy? Are they running old router/switch between?

When you have determined the location of your challenge, proceed to second step.

Second step
Second step of troubleshooting SharePoint Online performance is to measure ping time to the tenant as well as traceroute in order to locate route and geolocation of the tenant.
I use the following to complete that, either run it myself or ask the client to run it from their location.
psping.exe -n 20 tenant.sharepoint.com:443 > PsPingResult.txt
timeout 30 /nobreak
tracert tenant.sharepoint.com > TracertResult.txt
timeout 30 /nobreak

From that I derive the following:
1: For optimal performance, ping time is between 30-50ms stable.
2: Number of Route hops between 12-15.
3: Use https://www.iplocation.net/ to get most likely geolocation of tenant from returned public IP.

Psping.exe is part of SysInternals PSTools package.

Lastly again back to developer tools.
Collect performance report and look for errors in the console. Look for external http requests, certificate validation errors, script errors.

What the numbers show you
A: Ping times of more than 50MS or route hops more than 12-15 would lead me to look at the network with possible causes being:
Linespeed, old or outdated network equipment, tenant geolocation, proxy filtering, non-optimal routing.

B: Usually there are little or no problems in the direct network measurement and it will come down to the browser Development Tools. Note the difference between DOM loaded and Fully rendered. Client really dont care about DOM download, as they are only interested in the fully rendered page. So what can be causes of slow rendered pages:
Certificate errors, script errors, poor client hardware, proxy filtering, Antivirus, Poor coding.

How to configure Outgoing email in SharePoint with O365 – SMTP relay

How to configure Outgoing email in SharePoint with O365, SMTP relay.
You might have moved all your mail accounts to O365, but you still have that on-premises SharePoint server, that needs to send alerts or has some similar message functionality. Previously you had an Exchange server and used that as relay. Now you need to use O365, so how do you do that? Lets have a look at the prerequisites first and then I’ll show you how to put it all together to send messages, both internally and externally if required.

Prerequisites
– Service account in O365 with a mailbox; Used for authenticating SMTP request towards O365.
– Local SMTP server; Used for anonymous access to SharePoint SMTP.
– DNS record; Used as SMTP relay address internally.
– External IP address of local SMTP server; Used for SPF record registration.
– SPF record of mail domain; Used to validate the local SMTP server against public mail exchangers.
– Certificate that covers SMTP relay DNS address; This is used to provide required TLS encryption.
– Internal IP of SharePoint server(s); Used to allow the relay through local SMTP server.

SPF record
First thing to do is update your SPF record for your domain. This is done on your mail domains DNS settings and should be a text file.

SPF v=spf1 ip4: include:spf.protection.outlook.com ~all

Certificate
Install the matching certificate in the Personal store on the server.

Installation
If you do not have a local SMTP server already, you can install one using Roles and Features from within Windows Server.
To enable logging on the SMTP server, open IIS 6.0 Manager, expand your server and right click Properties. On the General tab; Check “Enable logging” and click Properties. Change log file directory to something different than your system drive.
On the Advanced Tab; Check the following Extended logging options:
Date (date), Time (time), Client IP Address (c-ip), Server Name (s-computername), Server IP Address (s-ip), Server Port (s-port), URI Query (cs-uri-query), Protocol Status (sc-status) and Protocol Substatus (sc-substatus).

Note: Take into consideration where you place the respective SMTP server folders. It is strongly recommended that you place them on a drive separate from the system drive.

IIS Configuration
Open IIS 6.0 Manager (which will be used to manage your SMTP server), expand your server and right click Properties on your SMTP Virtual Server.
On the access tab; under Secure communication it should state: “A TLS certificate is found with expiration date: “.
Click Authentication and verify that Anonymous access is enabled.
Click Relay, and select “Only the list below” and add the internal IP address of your SharePoint server(s). Leave the “Allow all computers which successfully authenticate to relay….” checked (this means that, all computers within the same domain may use this as a relay. IF you have infected machines, you want to disable this, or remove the infection).
Under the delivery tab; Click Outbound Security.
Check Basic authentication and type in your O365 service account information.
For example:

User name: svcRelayO365@contoso.com
Password: Ninja1234

Make sure TLS encryption is Checked and click Ok.
Click Outbound connections and change TCP port to 587 and click Ok.
Click Advanced, and type in the local DNS address of your internal relay and type in the SMART host smtp.office365.com and click Ok.
Example:

relay.contoso.com
smtp.office365.com

O365 Configuration
Login to portal.office365.com and navigate to Administration and Exchange.
In Office 365, click Admin, and then click Exchange to go to the Exchange Admin Center.
In the Exchange Admin Center, click Mail Flow, and click Connectors.
To add a new connector, click the + symbol and select From: “Your organization’s email server”, To: “Office 365” and click Next.
Choose the option “By verifying that the IP address of the sending server matches one of these IP addresses that belong to your organization”, and add the External IP address.
Leave all the other fields at their default values, and select Save.

SharePoint Configuration
Open Central Administation and click System Settings.
Click Configure outgoing e-mail settings.
Use the DNS name of your internal SMTP server as Outbound SMTP server and the From address should match that of your service account.

Testing & Troubleshooting
On your local SMTP server, create a file, called email.txt with the following content:

FROM:
TO:
SUBJECT: Test email
This is a test email sent from my SMTP server

Copy this file into the Pickup folder of your SMTP server. The server will process this and move it to the Queue folder and process it for delivery to O365.
If you do not receive an email at your personal email address within 5 minutes, something is wrong. Here is how to check.
Go to your log file directory, configured previously and have a look at the error codes provided there.
If they are “queued for delivery”, you move to the Office365 Portal and use the mailflow function and search for your mails. There they will be listed with a status indicating their state. The details of the Office365 mailflow log are comprehensive.

From SharePoint

 
$email = "test@test.com"
$subject = "Test subject"
$body = "Test body"
 
$site = New-Object Microsoft.SharePoint.SPSite "http://sharepoint"
$web = $site.OpenWeb()
[Microsoft.SharePoint.Utilities.SPUtility]::SendEmail($web,0,0,$email,$subject,$body)
 
// A True or False will confirm the message has been sent or not

References
https://technet.microsoft.com/en-us/library/dn554323%28v=exchg.150%29.aspx

http://jeffreypaarhuis.com/2013/02/12/send-test-email-from-sharepoint/

Windows 10 – Privacy settings – Automated

There has been quite some interest in the general privacy settings of the different applications and systems that we use in our daily life. Windows 10 is no exception to this, as Microsoft now by default, has chosen for you that you are very interested in sharing your traffic information with them. You can of course disable this tracking, however it is not always obvious how to do so. Once again my friend Mads Hjort Larsen has created a script to automate these privacy settings for you.

Have a look at the section “What should be changed” and make sure you change accordingly. ($false or $true)

Also note that once again WordPress has decided to mess with the formatting/syntax highlighting of the script. That is also why the third link in the comment section is surround by apostrophes.

<#
.SYNOPSIS
    This script automates the changing of a lot of settings that are otherwise hard or tedious to change.
.DESCRIPTION
    This script consolidates a lot of Windows registry changes and changes to other operating system settings, 
    that affect the appearance and functionality of the Windows 10 operating system, with the intent of making
    it more userfriendly and increasing the privacy.
.EXAMPLE
    win10privacy.ps1
.NOTES
    Created by Mads Hjort Larsen
    email: mads.hjort.larsen@gmail.com
.LINK
    
I made my own user-friendly Windows 10 privacy tool for inexperienced users. (Info in comments)
 byu/10se1ucgo inWindows10
.LINK
    http://pastebin.com/xq96nBGj
.LINK
    'https://gist.github.com/NickCraver/7ebf9efbfd0c3eab72e9/'
.LINK
    
My Windows 10 Tweaks

#>   

# What should be changed:
[bool]$privacySettings = $true  # change the settings related to privacy
[bool]$removeSoftware  = $false # remove Cortana, OneDrive and Metro Apps
[bool]$windowsUpdates  = $false # setup updates to NOT automatically reboot or download via P2P
[bool]$UIsettings      = $false # make win 10 look more like win 7/8/8.1

########### DO NOT EDIT BELOW THIS LINE #################################
Clear-Host

# http://blogs.technet.com/b/heyscriptingguy/archive/2011/05/11/check-for-admin-credentials-in-a-powershell-script.aspx 
If (-NOT ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")){
    Write-Warning "You do not have the Admin rights neccessary to run this script!`nPlease re-run this script as an Administrator!"
    Exit
}

# http://webcache.googleusercontent.com/search?q=cache:FjmfLRRqNb4J:https://fortheloveofcode.wordpress.com/2008/06/08/what-no-hkcr-in-powershell/+&cd=1&hl=en&ct=clnk&gl=dk
New-PSDrive -Name HKCR -PSProvider Registry -Root HKEY_CLASSES_ROOT

if($privacySettings -eq $true){
# Disable Cortana
New-ItemProperty -Path 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\Windows Search' -Name AllowCortana -PropertyType DWORD -Value 0 -ErrorAction SilentlyContinue

# Disable Data Logging Services
Get-Service diagtrack,dmwappushservice,RetailDemo | Stop-Service -PassThru | Set-Service -StartupType disabled
New-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection -Name AllowTelemetry -PropertyType DWORD -Value 0 -Force
New-ItemProperty -Path HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Policies\DataCollection -Name AllowTelemetry -PropertyType DWORD -Value 0 -Force

# Disable relevant scheduled tasks
schtasks /change /TN "\Microsoft\Windows\Application Experience\Microsoft Compatibility Appraiser" /DISABLE
schtasks /change /TN "\Microsoft\Windows\Application Experience\ProgramDataUpdater" /DISABLE
schtasks /change /TN "\Microsoft\Windows\Customer Experience Improvement Program\Consolidator" /DISABLE
schtasks /change /TN "\Microsoft\Windows\Customer Experience Improvement Program\KernelCeipTask" /DISABLE
schtasks /change /TN "\Microsoft\Windows\Customer Experience Improvement Program\UsbCeip" /DISABLE

# Erase the contents of AutoLogger-Diagtrack-Listener.etl
echo "" > C:\ProgramData\Microsoft\Diagnosis\ETLLogs\AutoLogger\AutoLogger-Diagtrack-Listener.etl

# Edit Hosts File
# http://www.dslreports.com/forum/r30222844-Stop-Windows-10-From-Spying-On-You-36-DNS-Addresses-to-host-file
$hostsPath = "$env:windir\System32\drivers\etc\hosts"
$hosts = get-content $hostsPath
[array]$urls = @(
    "a-0001.a-msedge.net"
    "a978.i6g1.akamai.net"
    "americas2.notify.windows.com.akadns.net"
    "any.edge.bing.com"
    "bl3302.storage.live.com"
    "bl3302geo.storage.dkyprod.akadns.net"
    "BN1WNS2011508.wns.windows.com"
    "choice.microsoft.com"
    "choice.microsoft.com.nsatc.net"
    "client.wns.windows.com"
    "compatexchange.cloudapp.net"
    "corp.sts.microsoft.com"
    "corpext.msitadfs.glbdns2.microsoft.com"
    "cs1.wpc.v0cdn.net"
    "df.telemetry.microsoft.com"
    "diagnostics.support.microsoft.com"
    "directory.services.live.com"
    "directory.services.live.com.akadns.net"
    "dns.msftncsi.com"
    "en-us.appex-rf.msn.com"
    "fe2.update.microsoft.com.akadns.net"
    "fe3.delivery.dsp.mp.microsoft.com.nsatc.net"
    "fe3.delivery.mp.microsoft.com"
    "feedback.microsoft-hohm.com"
    "feedback.search.microsoft.com"
    "feedback.windows.com"
    "i1.services.social.microsoft.com"
    "i1.services.social.microsoft.com.nsatc.net"
    "ipv6.msftncsi.com"
    "ipv6.msftncsi.com.edgesuite.net"
    "login.live.com"
    "login.live.com.nsatc.net"
    "oca.telemetry.microsoft.com"
    "oca.telemetry.microsoft.com.nsatc.net"
    "OneSettings-bn2.metron.live.com.nsatc.net"
    "pre.footprintpredict.com"
    "redir.metaservices.microsoft.com"
    "register.mesh.com"
    "reports.wes.df.telemetry.microsoft.com"
    "services.wes.df.telemetry.microsoft.com"
    "settings.data.glbdns2.microsoft.com"
    "settings-sandbox.data.microsoft.com"
    "settings-win.data.microsoft.com"
    "skyapi.live.net"
    "skyapi.skyprod.akadns.net"
    "skydrive.wns.windows.com"
    "sls.update.microsoft.com.akadns.net"
    "sqm.df.telemetry.microsoft.com"
    "sqm.telemetry.microsoft.com"
    "sqm.telemetry.microsoft.com.nsatc.net"
    "ssw.live.com"
    "ssw.live.com.nsatc.net"
    "statsfe1.ws.microsoft.com"
    "statsfe2.update.microsoft.com.akadns.net"
    "statsfe2.ws.microsoft.com"
    "survey.watson.microsoft.com"
    "telecommand.telemetry.microsoft.com"
    "telecommand.telemetry.microsoft.com.nsatc.net"
    "telemetry.appex.bing.net"
    "telemetry.appex.bing.net:443"
    "telemetry.microsoft.com"
    "telemetry.urs.microsoft.com"
    "travel.tile.appex.bing.com"
    "v10.vortex-win.data.metron.life.com.nsatc.net"
    "v10.vortex-win.data.microsoft.com"
    "vortex.data.microsoft.com"
    "vortex-sandbox.data.microsoft.com"
    "vortex-win.data.microsoft.com"
    "watson.live.com"
    "watson.microsoft.com"
    "watson.ppe.telemetry.microsoft.com"
    "watson.telemetry.microsoft.com"
    "watson.telemetry.microsoft.com.nsatc.net"
    "wes.df.telemetry.microsoft.com"
    "wildcard.appex-rf.msn.com.edgesuite.net"
    "win10.ipv6.microsoft.com"
    "win10.ipv6.microsoft.com.nsatc.net"
    "wns.notify.windows.com.akadns.net"
)

foreach($url in $urls){
    $hostLine = '0.0.0.0 '+$url

    if ($hosts -notcontains $hostLine){
        $newHosts = $hosts+$hostLine
        $newHosts | Out-File $hostsPath -Force
        }
 }

# WiFi Sense: HotSpot Sharing: Disable
Set-ItemProperty -Path HKLM:\Software\Microsoft\PolicyManager\default\WiFi\AllowWiFiHotSpotReporting -Name value -Type DWORD -Value 0

# WiFi Sense: Shared HotSpot Auto-Connect: Disable
Set-ItemProperty -Path HKLM:\Software\Microsoft\PolicyManager\default\WiFi\AllowAutoConnectToWiFiSenseHotspots -Name value -Type DWORD -Value 0

# Start Menu: Disable Bing Search Results
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Search -Name BingSearchEnabled -Type DWORD -Value 0

# Privacy: Disable Edge suggestions
Set-ItemProperty -Path HKL:\SOFTWARE\Policies\Microsoft\MicrosoftEdge\SearchScopes -Name ShowSearchSuggestionsGlobal -Type DWORD -Value 0

# Privacy: Let apps use my advertising ID: Disable
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\AdvertisingInfo -Name Enabled -Type DWORD -Value 0

# Privacy: SmartScreen Filter for Store Apps: Disable
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\AppHost -Name EnableWebContentEvaluation -Type DWORD -Value 0
}

if($removeSoftware -eq $true){
# Uninstall OneDrive
Stop-Process -ProcessName *OneDrive*
if(Test-Path "$env:windir\SysWOW64\OneDriveSetup.exe" -eq $true){Start-Process "$env:windir\SysWOW64\OneDriveSetup.exe" -ArgumentList '/uninstall' -Wait}
else{Start-Process "$env:windir\System32\OneDriveSetup.exe" -ArgumentList '/uninstall' -Wait}
Stop-Process -ProcessName *OneDrive* 

Remove-Item "%USERPROFILE%\OneDrive" -Recurse -Force
Remove-Item "C:\OneDriveTemp" -Recurse -Force
Remove-Item "%LOCALAPPDATA%\Microsoft\OneDrive" -Recurse -Force
Remove-Item "%PROGRAMDATA%\Microsoft OneDrive" -Recurse -Force

# Remove Cortana
Get-Process -Name *cortana* | Stop-Process
Get-AppxPackage -AllUsers | ? {$_.Name -match 'Cortana'} | Remove-AppxPackage -ErrorAction SilentlyContinue


#region Windows 10 Metro App Removals
# Be gone, heathen!
Get-AppxPackage king.com.CandyCrushSaga | Remove-AppxPackage

# Bing Weather, News, Sports, and Finance (Money):
Get-AppxPackage Microsoft.BingWeather | Remove-AppxPackage
Get-AppxPackage Microsoft.BingNews | Remove-AppxPackage
Get-AppxPackage Microsoft.BingSports | Remove-AppxPackage
Get-AppxPackage Microsoft.BingFinance | Remove-AppxPackage

# Xbox:
Get-AppxPackage Microsoft.XboxApp | Remove-AppxPackage

# Windows Phone Companion
Get-AppxPackage Microsoft.WindowsPhone | Remove-AppxPackage

# Solitaire Collection
Get-AppxPackage Microsoft.MicrosoftSolitaireCollection | Remove-AppxPackage

# People
Get-AppxPackage Microsoft.People | Remove-AppxPackage

# Groove Music
Get-AppxPackage Microsoft.ZuneMusic | Remove-AppxPackage

# Movies & TV
Get-AppxPackage Microsoft.ZuneVideo | Remove-AppxPackage

# OneNote
Get-AppxPackage Microsoft.Office.OneNote | Remove-AppxPackage

# Photos
Get-AppxPackage Microsoft.Windows.Photos | Remove-AppxPackage

# Sound Recorder
Get-AppxPackage Microsoft.WindowsSoundRecorder | Remove-AppxPackage

# Mail & Calendar
Get-AppxPackage microsoft.windowscommunicationsapps | Remove-AppxPackage

# Skype (Metro version)
Get-AppxPackage Microsoft.SkypeApp | Remove-AppxPackage

#endregion
}

if($windowsUpdates -eq $true){
# Change Windows Updates to "Notify to schedule restart"
# https://social.technet.microsoft.com/Forums/en-US/b8bf6607-99a0-441b-ab5f-f699ead7a56f/how-to-stop-windows-10-from-automatically-restarting?forum=WinPreview2014Feedback
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update' -Name AUOptions -Type DWORD -Value 4

# http://www.download3k.com/articles/How-to-Configure-Windows-Updates-in-Windows-10-01365
New-Item -Path HKLM:\SOFTWARE\Policies\Microsoft\Windows -Name WindowsUpdate
New-Item -Path HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate -Name AU 
New-ItemProperty -Path HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU -Name AUOptions -Type DWORD -Value 4 -Force

# Disable P2P Update downloads outside of local network
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\DeliveryOptimization\Config -Name DODownloadMode -Type DWORD -Value 1
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\DeliveryOptimization -Name SystemSettingsDownloadMode -Type DWORD -Value 3

# To disable P2P update downloads completely:
#Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\DeliveryOptimization\Config -Name DODownloadMode -Type DWORD -Value 0
}

if($UIsettings -eq $true){
# Change Explorer home screen back to "This PC"
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced -Name LaunchTo -Type DWORD -Value 1

# Disable Quick Access: Recent Files
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer -Name ShowRecent -Type DWORD -Value 0

# Disable Quick Access: Frequent Folders
Set-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer -Name ShowFrequent -Type DWORD -Value 0

# Disable the Lock Screen (the one before password prompt - to prevent dropping the first character)
If (-Not (Test-Path HKLM:\SOFTWARE\Policies\Microsoft\Windows\Personalization)) {New-Item -Path HKLM:\SOFTWARE\Policies\Microsoft\Windows -Name Personalization | Out-Null}
Set-ItemProperty -Path HKLM:\SOFTWARE\Policies\Microsoft\Windows\Personalization -Name NoLockScreen -Type DWORD -Value 1

# Use the Windows 7-8.1 Style Volume Mixer
If (-Not (Test-Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\MTCUVC")) {New-Item -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion" -Name MTCUVC | Out-Null}
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\MTCUVC" -Name EnableMtcUvc -Type DWORD -Value 0

# Remove folders from MyPC
# https://pricklytech.wordpress.com/2013/10/17/windows-8-1-x64-removing-the-folders-from-file-explorer/
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{1CF1260C-4DD0-4ebb-811F-33C572699FDE}" -Force # Music
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{374DE290-123F-4565-9164-39C4925E467B}" -Force # Downloads
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{3ADD1653-EB32-4cb0-BBD7-DFA0ABB5ACCA}" -Force # Pictures
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{A0953C92-50DC-43bf-BE83-3742FED03C9C}" -Force # Videos
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{A8CDFF1C-4878-43be-B5FD-F8091C1C60D0}" -Force # Documents

Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{088e3905-0323-4b02-9826-5d99428e115f}" -Force # Downloads 
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{24ad3ad4-a569-4530-98e1-ab02f9417aa8}" -Force # Pictures
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{3dfdf296-dbec-4fb4-81d1-6a3438bcf4de}" -Force # Music
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{B4BFCC3A-DB2C-424C-B029-7FE99A87C641}" -Force # Desktop
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{d3162b92-9365-467a-956b-92703aca08af}" -Force # Documents
Remove-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MyComputer\NameSpace\{f86fa3ab-70d2-4fc7-9c99-fcbf05467f3a}" -Force # Videos
 
# Remove OneDrive from the Explorer Side Panel.
Remove-Item "HKCR:\CLSID\{018D5C66-4533-4307-9B53-224DE2ED1FE6}" -Force -Recurse
Remove-Item "HKCR:\Wow6432Node\CLSID\{018D5C66-4533-4307-9B53-224DE2ED1FE6}" -Force -Recurse

# Remove Compressed file/folder
Remove-Item "HKCR:\CABFolder\CLSID" -Force -Recurse
Remove-Item "HKCR:\CompressedFolder\CLSID" -Force -Recurse
Remove-Item "HKCR:\SystemFileAssociations\.cab\CLSID" -Force -Recurse
Remove-Item "HKCR:\SystemFileAssociations\.zip\CLSID" -Force -Recurse

# Explorer: Show all folders
New-ItemProperty -path HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced -Name NavPaneShowAllFolders -PropertyType DWORD -Value 1

# Explorer: Show file extensions
New-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced -Name HideFileExt -PropertyType DWORD -Value 0

# Explorer: Show hidden files
New-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced -Name Hidden -PropertyType DWORD -Value 1

# Taskbar: Show all icons in taskbar
New-ItemProperty -Path HKLM:\Software\Microsoft\Windows\CurrentVersion\Explorer -Name EnableAutoTray -PropertyType DWORD -Value 0
New-ItemProperty -Path HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced -Name NavPaneShowAllFolders -PropertyType DWORD -Value 1

# Taskbar: Hide Task View Button
New-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced -Name ShowTaskViewButton -PropertyType DWORD -Value 0

# Taskbar: Hide Search
New-ItemProperty -Path HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Search -Name SearchboxTaskbarMode -PropertyType DWORD -Value 0

# Bring back old Windows Update control panel app
# http://answers.microsoft.com/en-us/insider/forum/insider_wintp-insider_update/need-the-old-windows-update-not-the-new-windows/35bc83a7-3aa9-4408-b189-4aa2777e4e11
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\WindowsUpdate\UX -Name IsConvergedUpdateStackEnabled -Value 0 -Force
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\WindowsUpdate\UX\Settings -Name UxOption -Value 0 -Force

# Titlebar: Enable accent colors
# http://www.intowindows.com/how-to-change-title-bar-color-in-windows-10/ 
Copy-Item -Path $env:windir\Resources\Themes\aero -Recurse -Destination $env:windir\Resources\Themes\windows -Force -ErrorAction SilentlyContinue
Get-ChildItem -Path $env:windir\Resources\Themes\windows -Filter "aero.msstyles*" -Recurse | Rename-Item -NewName {$_.name -replace 'aero','windows' }
(Get-Content $env:windir\Resources\Themes\aero.theme).Replace('Path=%ResourceDir%\Themes\Aero\Aero.msstyles', 'Path=%ResourceDir%\Themes\windows\windows.msstyles') | Set-Content $env:TEMP\windows.theme
Start-Process $env:TEMP\windows.theme -Wait
(New-Object -comObject Shell.Application).Windows() | where-object {$_.LocationName -eq "Personalization"} | foreach-object {$_.quit()}
Remove-Item $env:TEMP\windows.theme
}

# Finishing touches
Write-Host "Please make sure you do the following as well:`nSelect Never in the first box, and Basic in the second box"
start ms-settings:privacy-feedback