SharePoints

SharePoint Online – Performance – Basic Troubleshooting

Classic case: Customer reports in “SharePoint Online performance is slow. [period]”
Account Manager, Product Manager, Project Manager comes running 4 weeks after due date, customer wont accept the solution with the given performance. Ok…. Lets gather the tools for first steps:

Toolbox includes at this point
– PSping utility from PSTools(PSTools )
– Tracert utility from Microsoft (TraceRT )
– Browser Development Tools (F12 for Internet Explorer, Firefox or Chrome)

First step
Compare performance from your location with the client location using the Browser development tools. Are they the same?
If yes, problem usually lies in the configuration of SharePoint Online. Start testing from your location and don’t bother asking customer for client computer, remote login, network information.
If no, problem usually lies from customer client machine and SharePoint Online server. Start testing from customer client machine at their location. Ask them for information regarding their network configuration. Do they have proxy? Are they running old router/switch between?

When you have determined the location of your challenge, proceed to second step.

Second step
Second step of troubleshooting SharePoint Online performance is to measure ping time to the tenant as well as traceroute in order to locate route and geolocation of the tenant.
I use the following to complete that, either run it myself or ask the client to run it from their location.
psping.exe -n 20 tenant.sharepoint.com:443 > PsPingResult.txt
timeout 30 /nobreak
tracert tenant.sharepoint.com > TracertResult.txt
timeout 30 /nobreak

From that I derive the following:
1: For optimal performance, ping time is between 30-50ms stable.
2: Number of Route hops between 12-15.
3: Use https://www.iplocation.net/ to get most likely geolocation of tenant from returned public IP.

Psping.exe is part of SysInternals PSTools package.

Lastly again back to developer tools.
Collect performance report and look for errors in the console. Look for external http requests, certificate validation errors, script errors.

What the numbers show you
A: Ping times of more than 50MS or route hops more than 12-15 would lead me to look at the network with possible causes being:
Linespeed, old or outdated network equipment, tenant geolocation, proxy filtering, non-optimal routing.

B: Usually there are little or no problems in the direct network measurement and it will come down to the browser Development Tools. Note the difference between DOM loaded and Fully rendered. Client really dont care about DOM download, as they are only interested in the fully rendered page. So what can be causes of slow rendered pages:
Certificate errors, script errors, poor client hardware, proxy filtering, Antivirus, Poor coding.

How to configure Outgoing email in SharePoint with O365 – SMTP relay

How to configure Outgoing email in SharePoint with O365, SMTP relay.
You might have moved all your mail accounts to O365, but you still have that on-premises SharePoint server, that needs to send alerts or has some similar message functionality. Previously you had an Exchange server and used that as relay. Now you need to use O365, so how do you do that? Lets have a look at the prerequisites first and then I’ll show you how to put it all together to send messages, both internally and externally if required.

Prerequisites
– Service account in O365 with a mailbox; Used for authenticating SMTP request towards O365.
– Local SMTP server; Used for anonymous access to SharePoint SMTP.
– DNS record; Used as SMTP relay address internally.
– External IP address of local SMTP server; Used for SPF record registration.
– SPF record of mail domain; Used to validate the local SMTP server against public mail exchangers.
– Certificate that covers SMTP relay DNS address; This is used to provide required TLS encryption.
– Internal IP of SharePoint server(s); Used to allow the relay through local SMTP server.

SPF record
First thing to do is update your SPF record for your domain. This is done on your mail domains DNS settings and should be a text file.

SPF v=spf1 ip4: include:spf.protection.outlook.com ~all

Certificate
Install the matching certificate in the Personal store on the server.

Installation
If you do not have a local SMTP server already, you can install one using Roles and Features from within Windows Server.
To enable logging on the SMTP server, open IIS 6.0 Manager, expand your server and right click Properties. On the General tab; Check “Enable logging” and click Properties. Change log file directory to something different than your system drive.
On the Advanced Tab; Check the following Extended logging options:
Date (date), Time (time), Client IP Address (c-ip), Server Name (s-computername), Server IP Address (s-ip), Server Port (s-port), URI Query (cs-uri-query), Protocol Status (sc-status) and Protocol Substatus (sc-substatus).

Note: Take into consideration where you place the respective SMTP server folders. It is strongly recommended that you place them on a drive separate from the system drive.

IIS Configuration
Open IIS 6.0 Manager (which will be used to manage your SMTP server), expand your server and right click Properties on your SMTP Virtual Server.
On the access tab; under Secure communication it should state: “A TLS certificate is found with expiration date: “.
Click Authentication and verify that Anonymous access is enabled.
Click Relay, and select “Only the list below” and add the internal IP address of your SharePoint server(s). Leave the “Allow all computers which successfully authenticate to relay….” checked (this means that, all computers within the same domain may use this as a relay. IF you have infected machines, you want to disable this, or remove the infection).
Under the delivery tab; Click Outbound Security.
Check Basic authentication and type in your O365 service account information.
For example:

User name: svcRelayO365@contoso.com
Password: Ninja1234

Make sure TLS encryption is Checked and click Ok.
Click Outbound connections and change TCP port to 587 and click Ok.
Click Advanced, and type in the local DNS address of your internal relay and type in the SMART host smtp.office365.com and click Ok.
Example:

relay.contoso.com
smtp.office365.com

O365 Configuration
Login to portal.office365.com and navigate to Administration and Exchange.
In Office 365, click Admin, and then click Exchange to go to the Exchange Admin Center.
In the Exchange Admin Center, click Mail Flow, and click Connectors.
To add a new connector, click the + symbol and select From: “Your organization’s email server”, To: “Office 365” and click Next.
Choose the option “By verifying that the IP address of the sending server matches one of these IP addresses that belong to your organization”, and add the External IP address.
Leave all the other fields at their default values, and select Save.

SharePoint Configuration
Open Central Administation and click System Settings.
Click Configure outgoing e-mail settings.
Use the DNS name of your internal SMTP server as Outbound SMTP server and the From address should match that of your service account.

Testing & Troubleshooting
On your local SMTP server, create a file, called email.txt with the following content:

FROM:
TO:
SUBJECT: Test email
This is a test email sent from my SMTP server

Copy this file into the Pickup folder of your SMTP server. The server will process this and move it to the Queue folder and process it for delivery to O365.
If you do not receive an email at your personal email address within 5 minutes, something is wrong. Here is how to check.
Go to your log file directory, configured previously and have a look at the error codes provided there.
If they are “queued for delivery”, you move to the Office365 Portal and use the mailflow function and search for your mails. There they will be listed with a status indicating their state. The details of the Office365 mailflow log are comprehensive.

From SharePoint

 
$email = "test@test.com"
$subject = "Test subject"
$body = "Test body"
 
$site = New-Object Microsoft.SharePoint.SPSite "http://sharepoint"
$web = $site.OpenWeb()
[Microsoft.SharePoint.Utilities.SPUtility]::SendEmail($web,0,0,$email,$subject,$body)
 
// A True or False will confirm the message has been sent or not

References
https://technet.microsoft.com/en-us/library/dn554323%28v=exchg.150%29.aspx

http://jeffreypaarhuis.com/2013/02/12/send-test-email-from-sharepoint/

Disable unsafe ciphers and SSL 2.0/3.0 on your server

One step to improve the security on your servers, would be to disable SSL 2.0 and 3.0 as well as the unsafe ciphers RC4. This can be done using the following registry changes on your server.
Note: When you disable SSL 2.0 and 3.0 on your servers, clients will no longer be able to connect using that. See this post for additional reference.

RC4 Cipher

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\RC4 128/128]
"Enabled"=dword:00000000
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\RC4 40/128]
"Enabled"=dword:00000000
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\RC4 56/128]
"Enabled"=dword:00000000

SSL 2.0 and 3.0

Windows Registry Editor Version 5.00
[HKey_Local_Machine\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0]
"Enabled"=dword:00000000
[HKey_Local_Machine\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0]
"Enabled"=dword:00000000

References:
https://en.wikipedia.org/wiki/RC_algorithm
https://jesperarnecke.wordpress.com/2014/04/24/web-server-security-ssltls/
https://technet.microsoft.com/en-us/library/dn786418.aspx

SharePoint – ConfigDB – Growing

Alright, so one of my developers was complaining that his config database on his development environment was more than 40GB(If your config database is larger than 10GB, you should continue reading), or actually it was the sysadmin that complained, but my developer was targeted. He asked if I could have a look at it. Sure of course, properly transaction logs gone wild…. So logging on the server, finding the trans logs were all good. All right further digging came up with the TimerJobHistory table being filled with around 100 million records. Alright, now what?

Some posts have some good Powershell scripts, that will incrementally delete your timerjob history, maybe also change your retention time. Sure I started to run those, but that didn’t quite do the trick. So what was wrong then?

Alright here is the deal. The “Delete Job History” job, has a timeout of 5 minutes. That means it will delete records, but if the delete is not completed within 5 minutes, it throws a timeout. It actually fails the timer job. (Timeout) This timer job is set to run only once a week. So guess what happens when you create more timer jobs per week than the Delete Timer job can remove in 5 minutes? – Yup, the TimerJobHistory table will grow…

I found the easiest fix, just to change the schedule for the “Delete Job History”. For the 100 million records, I changed it to every 7’th minute. I know that it will not run longer than 5 minutes, so a 2 minutes slack should be sufficient. So what should the setting be? Adjust according to your environment.

Summary
SharePoint ConfifDB growing beyond the normal 4-8GB. The data file, not the log file.

Root cause
The Delete Job History cannot delete more rows than created timer jobs per week within the 5 minute timeout. This can be related to heavy deployment or insufficient resources.

Solution
Change the schedule of the Delete Job History to run more frequently.

Powershell
Get-SPTimerJob job-Delete-Job-History | Set-SPTimerJob -Schedule “daily at 05:00:00”

Which interval is required depends on your environment or the amount of rows in backlog. So set schedule accordingly and following the types for Set-SPTimerJob.
The type must be a valid SharePoint Timer service (SPTimer) schedule in the form of any one of the following schedules:
– Every 5 minutes between 0 and 59
– Hourly between 0 and 59
– Daily at 15:00:00
– Weekly between Fri 22:00:00 and Sun 06:00:00
– Monthly at 15 15:00:00
– Yearly at Jan 1 15:00:00

Cookies – IE – ADFS – MSIS7001

Recently we had some strange issues with an ADFS login. Everything worked, but it didn’t. On some sites we got the following error:

“MSIS7001: The passive protocol context was not found or not valid. If the context was stored in cookies, the cookies that were presented by the client were not valid. Ensure that the client browser is configured to accept cookies from this website and retry this request”

One of my colleagues pointed out that all sites with the error contained the underscore (“_”) character, so we started digging into this and found out indeed IE has some issues with the underscore chars in the URL. More accurately the way IE is designed, makes it incapable of creating cookies, if the URL contains an underscore in the domain name.
We found this QandA on Internet Explorer and cookies:
http://blogs.msdn.com/b/ieinternals/archive/2009/08/20/wininet-ie-cookie-internals-faq.aspx

Snowballing further from that link leads to the following KB:
https://support.microsoft.com/kb/316112/

“Security patch MS01-055 prevents servers with improper name syntax from setting cookies names. Domains that use cookies must use only alphanumeric characters (“-” or “.”) in the domain name and the server name. Internet Explorer blocks cookies from a server if the server name contains other characters, such as an underscore character (“_”).
Because ASP session state and session variables rely on cookies to function, ASP cannot maintain session state between requests if cookies cannot be set on the client.
This issue can also be caused by an incorrect name syntax in a host header.”

Basically the above security patch is implemented as part of Internet Explorer and the way it handles domain names and cookies.
So far this has been tested on the following versions of of Internet Explorer; IE8, IE9, IE10 and IE11.
This is not a problem for Chrome or Firefox – I have not tested with other browsers or versions.

Adding Users/Groups – SharePoint – Powershell

A few times I’ve had to add users to specific SharePoint groups using powershell. I made the below script, which splits up each of the processes in the user creation and permission handling into transparent chunks. That way it’s easier to take what you need 🙂
The below users are external identity provider users, based on UPN. There is a domain users group being added also. The rest of the code should be self explanatory.

#Defines the site to work with
$URL= 'https://intranet.contoso.com/HR' 

#Gets the required web and site objects to work with
$Site= Get-SPSite $URL
$Web=$Site.RootWeb

#Creating Users
$JD=get-spweb $url | New-SPUser -UserAlias 'i:0e.t|Azure ACS|John.Doe@live.com'
$ON=get-spweb $url | New-SPUser -UserAlias 'i:0e.t|Azure ACS|Ola.Nordmann@live.com'
$AA=get-spweb $url | New-SPUser -UserAlias 'i:0e.t|Azure ACS|Anders.Andersen@gmail.com'
$MS=get-spweb $url | New-SPUser -UserAlias 'i:0e.t|Azure ACS|Medel.Svensson@contoso.com'

#Creating Groups
$DUContoso=get-spweb $url | New-SPUser -UserAlias 'c:0-.t|Azure ACS|Contoso\Domain Users'

#Get site default groups (using just "$web.Sitegroups" will show all of them.)
$HROwn=$web.SiteGroups["HR Owners"]
$HRMem=$web.SiteGroups["HR Members"]
$HRVis=$web.SiteGroups["HR Visitors"]

#Adding Users to groups
#Owners
$HROwn.AddUser($AA)
$HROwn.AddUser($MS)

#Members
$HRMem.AddUser($ON)

#Visitors
$HRVis.AddUser($DUContoso)