Rename powershell with case sensitive

I ran this on powershell/linux to fix some case sensitive file names. I know that's a weird combo but I can't remember any bash.

$files = Get-ChildItem -Name *.PNG -recurse -Path ./ | grep .PNG foreach($file in $files) { $newName = $file -replace '.PNG','.png' git mv $file $newName; }

Sure it not pretty but it works for me!

SPN for network service

“

The HOST service represents the host computer. The Kerberos protocol uses the HOST SPN to access the host computer. The Kerberos protocol uses the long-term key on the host computer to create a service ticket.
The HTTP service class is one of the built-in services that act as an alias to the HOST SPN. The HOST SPN is mapped to the host computer account. Therefore, when you use the default HTTP service class, the Kerberos protocol uses the computer account as the service account to request a service ticket.”

https://support.microsoft.com/en-gb/help/929650/how-to-use-spns-when-you-configure-web-applications-that-are-hosted-on

Translation – if you are using network service and tying to set the SPN – use the computer name as the account name.

Some simple secure development standards

These are mainly based around ASP.NET/IIS – but the same principles can be applied everywhere.

These are just a few that I’ve seen highlighted a lot in pen tests of late. 

Nothing is going to make you 100% secure but remember – defence in depth.  Lots of defences adds up.

I may get some others written up some time.

(HTML/URL) Encode everything

Why

This helps prevent Cross Site Scripting (XSS) attacks.  XSS attacks occur when user input (or other data) is displayed on the browser.  The attacker puts javascript in the input and this is then executed by the browser. 

Examples

sending the user a link with a query string which contains a value that is displayed on the target page – look at google and you’ll see the search is in the url and displayed on the browser.  (Reflected XSS).

Putting data in to the system that is displayed on someone elses browser (e.g. facebook page) (Stored XSS).  For a great example of this watch this watch  https://www.youtube.com/watch?v=EYMGAoIx8yk&feature=youtu.be FROM 8:30

How?

• Http(s) content type MUST be utf-8 – for asp.net this can be specified in the master page / web.config.
• In ASP.NET (>4) Use the <%#:, <%:, <%=: to ensure databound or other expressions are html encoded (https://www.owasp.org/index.php/ASP.NET_Output_Encoding).
• Use ASP.NET controls that automatically html encode/decode. See - https://msdn.microsoft.com/en-us/library/ms178270%28v=VS.100%29.aspx for an indication of which controls automatically html encode output.  In particular note the Label Control does not automatically encode.
• Razor automatically html encodes (https://docs.microsoft.com/en-us/aspnet/web-pages/overview/ui-layouts-and-themes/4-working-with-forms) – think carefully before overriding.  (You might have actual html that you want to display as html).

Use strongly typed parameterised queries (that’s stored procs or EF).

Why

To mitigate against SQL injection attacks.

Parameterised queries (and stored procs) should treat the input as input – not build sql strings from them so cannot be manipulated as easily.

How?

Use an ORM (Entity Framework) AND DO NOT GENERATE SQL ON THE FLY IN YOUR CODE.

Use strongly typed stored procedures and DO NOT GENERATE SQL ON THE FLY IN YOUR SQL.

Combine this with

• least privilege (execute on stored procs or data reader/writer on appropriate tables for EF) – note SPROCS allow a more minimal access as an attaker owning the applications connection can still not read whole tables but needs to go through stored procs.
• A defence mechanism for preventing developers from still generating sql on the fly.
• e.g. for stored procs review them all.

EF does not particularly encourage sql on the fly but none the less be aware application developers now have a connection available that can do data reader/writer functions.  I have seen this result in a successful attack – not surprisingly on the last (hastily implemented) CR implementation.

Implement some secure configuration/hardening

These are some simple hardening recommendations – may not do much but will not make you less secure!

Why?

• Your application and server (IIS) etc. uses technologies.  These may have known vulnerabilities.  Advertising the technologies is the same as advertising the weaknesses.  Security by obscurity is not security but giving attackers a list of sites to attack when a vulnerability is published is dumb.
• Reduce the attack surface area.

How?

•Disclosure of server type / Disclosure of technology

* For azure - (https://azure.microsoft.com/en-us/blog/removing-standard-server-headers-on-windows-azure-web-sites/) – this should work in IIS too – but depends on later frameworks.

For IIS - https://www.saotn.org/remove-iis-server-version-http-response-header/

• Block unwanted request verbs  -  for a web site you probably only need GET and POST.  For rest you might need PUT and DELETE too (if your really rest). https://docs.microsoft.com/en-us/iis/configuration/system.webserver/security/requestfiltering/verbs/  Personally, of course, I prefer white-listing

<configuration> <system.webServer> <security> <requestFiltering> <verbs allowUnlisted="false" > <add verb="GET" allowed="true" />
     <add verb="POST" allowed="true" /></verbs>
   </requestFiltering>
  </security>
</system.webServer>
</configuration>

See - https://docs.microsoft.com/en-us/iis/manage/configuring-security/use-request-filtering

• Set custom errors RemotOnly (https://msdn.microsoft.com/en-us/library/h0hfz6fc(v=vs.100).aspx)
• Set debug=false.

Use least privilege database access (or any other access!)

Why?

Defence in depth – if someone owns your web server or another defence (e.g. sql injection) fails – limits the damage can be done.

How

• Don’t use dbowner/sa
• Do use separate account for applications with privilege to execute stored procs (GRANT EXEC ON proc TO user) or (if using EF) data reader/writer as required on specific tables.
• Do this from the beginning in all your environments so you don’t hit all the issues late in your dev cycle.

Configure your security headers and cookies

See - https://securityheaders.com/

Why?

Modern browsers are adding additional protection against vulnerabilities – xss, disclosure.

How?

See https://securityheaders.com/

If your are https (and you should be)

Enable hsts

<system.webServer>

<httpProtocol>

<customHeaders>

<add name="Strict-Transport-Security" value="max-age=31536000"/>

</customHeaders>

</httpProtocol>

</system.webServer>

Set requires SSL on all your cookies - <httpCookies requireSSL="true" />

(and your forms auth config too if you have it).

X-Frame-Options – do you want your pages to be embedded in another site.  Allowing this facilitates an attacker making a fake site.

X-XSS-Protection – tells the browser to turn on xss filters.

X-Content-Type-Options – stops content type sniffing

Content Security Policy – allows you to list where your page downloads from so you can protect from downloading malicious code.

Referrer Policy – limits the “Referrer” information in http requests.  For example, if you download fonts from google do you want google to be able to track people on your site through the referrer header.  Remember you may letting a third party know the IP (personal information under GDPR) the activity on your site.

Azure–iterate resources in all tenants for GDPR

GDPR means that we now have a policy for all data to be in UK or Europe.

So – to find where all our azure resources are

1. Install the azure PowerShell toolkik (https://docs.microsoft.com/en-gb/azure/azure-resource-manager/powershell-azure-resource-manager)

2. Run the script below to check the location of your resources are in the list of allowed locations.

# login to azure this should request secure credentials
Login-AzureRmAccount
# get a list of tenants / subscriptions
$allowedlocations = 'northeurope', 'westeurope' , 'francecentral' , 'francesouth', 'ukwest' , 'uksouth', 'germanycentral' , 'germanynortheast'
$subscriptions = Get-AzureRmSubscription
foreach($sub in $subscriptions) {
     Select-AzureRmSubscription -Subscription $sub.Id
     $resources = Get-AzureRmResource
     $resources.where({ $_.Location  -notin $allowedlocations }) 
}

Strava open image script

Sometimes I want to download peoples photos from strava.

Click on image to get it full screen

run

$im = $('div[class="photo-slideshow-content"] > image[alt="Photo"]').src
window.open($im)

in console.

Will open the image in a new window so you can download.

SSRS (2014) Load balancing woes

Symptoms

HTTPS to reporting services only working on prd-web01b, not on 01a.  Checked all  the config etc. and re imported certs from b to a.  SChannell errors were our only clue, although William found some of these on the other server too.  Even rebooted the server for good measure.

How we fixed it

This was the clue - https://support.microsoft.com/en-gb/help/956209/ssl-no-longer-works-after-you-remove-an-ssl-binding-from-sql-server-20

We added the cert into IIS (even though we are running in native mode) , removed it from IIS and rebound it to reporting services and everything now appears ok.

Explanation

Some months ago we upgraded all our certs – I think it was to 2056 bits or such like – as the old ones were becoming invalid.

My theory is on the a server we unbound the old cert – thus removing the crucial registry setting in the above link – and then bound in the new cert.

On the b server we probably just selected the new cert.

I’m not sure who did this work but since a good manager always takes the blame for their teams actions – it was probably me.  (Lesson – sooner we go to scripted deploy the better).

However this did not fix the problem!

2’nd problem

The ever clever mr***suggested looking at the logs – and I found them – FYI there in d:\ Program Files\Microsoft SQL Server\MSRS12.MSSQLSERVER\Reporting Services\LogFiles on our servers.

Found the errors –

library!ReportServer_0-36!1050!09/05/2017-08:54:29:: e ERROR: Error rending control: System.Web.HttpException: Validation of viewstate MAC failed. If this application is hosted by a Web Farm or cluster, ensure that <machineKey> configuration specifies the same validationKey and validation algorithm. AutoGenerate cannot be used in a cluster.

http://go.microsoft.com/fwlink/?LinkID=314055 ---> System.Web.UI.ViewStateException: Invalid viewstate.

                Client IP: *****

                Port: 59222

                User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko

                ViewState: /wEPDwUKMTEzMjExOTUxNA9kFgQCAQ8WAh4EbGFuZwUFZW4tR0JkAgMPZBYEAgQPZBYEAgEPFgIeBVZhbHVlZGQCAw9kFgJmD2QWAmYPFgIfAWRkAgUPFCsABQ8WCh4UU2hvd1Byb21wdEFyZWFCdXR0b25oHgxTY3JvbGxUYXJnZXRkHhNQcm9tcHRBcmVhQ29sbGFwc2VkZx4QVjFTdHlsZVNoZWV0TmFtZWQeDlJlbmRlcmluZ1N0YXRlCymRAU1pY3Jvc29mdC5SZXBvcnRpbmcuV2ViRm9ybXMuUmVwb3J0UmVuZGVyaW5nU3RhdGUsIFJlcG9ydGluZ1NlcnZpY2VzV2ViU2VydmVyLCBWZXJzaW9uPTEyLjAuMC4wLCBDd

..

How we fixed it

This was easy for a web farm person like me – when you have a .net application  in a web farm you need to add in encryption keys across the farm.

First I checked the instructions as I would have thought that SSRS might do this for me as you actually configure the thing for scale out but MS aren’t that bright – yes you do need to manually put in some keys (https://docs.microsoft.com/en-us/sql/reporting-services/report-server/configure-a-report-server-on-a-network-load-balancing-cluster).

So I generated some for each environment and did this.

Explanation

I always said the old load balancer was not load balancing.  The new ones (F5) are. 

Import and bind a cert is IIS.

param (
[Parameter(Mandatory=$true)][String]$certpath,
[Parameter(Mandatory=$true)][String]$certpass,
[Parameter(Mandatory=$true)][String]$ip,
[Parameter(Mandatory=$false)][String]$website = "Default Web Site"
)

$mypwd = ConvertTo-SecureString -String $certpass -Force –AsPlainText
$cert  = Import-PfxCertificate -FilePath $certpath Cert:\LocalMachine\My -Password $mypwd -Exportable
$bind = Get-WebBinding -Protocol https -IPAddress $ip  -port 443
if($bind -ne $null) {
    Remove-WebBinding  -Protocol https -IPAddress $ip  -port 443
}

# -Name $website -IPAddress $ip -Protocol HTTPS -Port 443 -HostHeader '' -Binding $ip':433:'
new-WebBinding -Name $website -IPAddress $ip -Protocol https -Port 443 
$bind = Get-WebBinding -Protocol https -IPAddress $ip  -port 443
$bind.AddSslCertificate($cert.GetCertHashString(), "my")

This is for a cert to IP without host header.

Stop, start disable services on multiple machines

$Machines = Get-Content -Path ".\Machines.txt"
$service="Microsoft Deployment Agent"
$credential = Get-Credential
foreach($computer in $Machines) {
    #$result = (gwmi win32_service -computername $computer -filter "name='$service'" -Credential $credential).stopservice() 
    #$result = (gwmi win32_service -computername $computer -filter "name='$service'" -Credential $credential).ChangeStartMode("Disabled") 
    $result = (gwmi win32_service -computername $computer -filter "name='$service'" -Credential $credential).startservice()     
    $result = (gwmi win32_service -computername $computer -filter "name='$service'" -Credential $credential).ChangeStartMode("Automatic") 
}
# $result = (gwmi win32_service -computername $computer -filter "name='$service'" -Credential $cred).startservice() 
#Get-Service -Name $Services -ComputerName $Machines -Credential $credential | Set-Service -Status Started -StartupType Automatic 

As usual not really my code but plagiarised. Massive apologies to whoever/wherever I got it as I can’t find it again.  As usual – this is really for me!

The Get-Service /  Set-Service script was the obvious candidate to use but wont take a credential.

Put a list of machines in Machines.txt.

Original that I’ve lost also took a list of services!

Grep for PowerShell

Get-Childitem -Path ".\" web.config -Recurse | Select-String -pattern 'dardaudit' –SimpleMatch

Works for me!

Resetting the product.

OK – so my audax applicaiton has not received an update in 4 months and I have just completed the paperwork for the Titanic Torr 2017.

Failed.

Thinking about it on my bike the MVP was far to big.

All I really want is a list of entrants details – that’s the MVP – one page – fill in the form.

To close for the North to the South and only a select few idiots elite riders do this but Antrim 300 will be a target!

WCF endpoints authentiction with HTTP and HTTPS

Add in a new endpoint

<endpoint contract="PortalServices.IMenuService" binding="webHttpBinding" address="restService" behaviorconfiguration="restJsonBehaviour" bindingconfiguration="WindowsHttpBindingHttps" />

With a binding confuguration for https with transport security mode.

<binding name="WindowsHttpBindingHttps">
          <security mode="Transport">
            <!-- Needs to be "TransportCredentialOnly" for Windows and "Transport" for https -->
            <transport clientcredentialtype="Windows" />
          </security>        
        </binding>

Enable https on the binding

<servicemetadata httpgetenabled="true" httpsgetenabled="true" />     

 

KTA permissions for service accounts

Don’t ask – but heres a script to set permissions for KTA service accounts to run as non admins.

They also need lon on as service account permissions.

script to grant folder permission from here - http://techibee.com/powershell/grant-fullcontrol-permission-to-usergroup-on-filefolder-using-powershell/2158

param (

       [Parameter(Mandatory=$true)][string]$serviceAccount

) 



function Grant-userFullRights( [string[]]$Files, [string]$UserName) {           
 $rule=new-object System.Security.AccessControl.FileSystemAccessRule($UserName,"FullControl","Allow")            

 foreach($File in $Files) {            
  if(Test-Path $File) {            
   try {            
    $acl = Get-ACL -Path $File -ErrorAction stop            
    $acl.SetAccessRule($rule)            
    Set-ACL -Path $File -ACLObject $acl -ErrorAction stop            
    Write-Host "Successfully set permissions on $File"            
   } catch {            
    Write-Warning "$File : Failed to set perms. Details : $_"            
    Continue            
   }            
  } else {            
   Write-Warning "$File : No such file found"            
   Continue            
  }            
 }            
}
[string]$UserName = $serviceAccount

$Files = @("C:\ProgramData\Kofax\AppLogging\DB", "C:\ProgramData\Kofax Image Products\Local\Scripts")

Grant-userFullRights $Files $UserName
netsh http add urlacl http://+:80/Agility.Sdk.Services.StreamingService user=$UserName
netsh http add urlacl  http://+:3581/SALMetadata/ user=$UserName
netsh http add urlacl  http://+:3581/SAL/ user=$UserName
net stop "TotalAgility Streaming Service"
net start "TotalAgility Streaming Service"
net stop "KSALicenseService"
net start "KSALicenseService"

Powershell endpoints

For old school .svc and .asmx

foreach($dir in ("dir1",”dir2”)) {
    $files = Get-ChildItem -Path  D:\AppWebSites\$dir -Recurse -Include ('*.asmx','*.svc')
    $files
}

Quick event logging guide with MS EL–really for me

CONCEPTS

· Source = Only used by the Machine Event Viewer

· EventId  number that goes in the EventId column of database Log

· TraceEventType (System.Diagnostics.TraceEventType) = a system enum for the log level (info, verbose, error, critical)

PROJECT

We need to add a reference to these:

Config

This page explains it - https://msdn.microsoft.com/en-us/library/ff664760(v=pandp.50).aspx

Where is the tool?

The tool is in tfs - /EnterpriseLibrary5/Bin/EntLibConfig.exe

 

REGISTERING SOURCES FOR EVENTS LOG

If you ever need to register a source (for the Event Log), you run this from PowerShell

New-EventLog -LogName Application -SRC MyNewSource –computername <server>,<other>,<servers,go,here>

Is my source registered?

There’s a way to know what sources has been registered in a machine (see attached).

But it is easier to simply run the previous command to make sure

ODBC Settings multuple servers

You need to have some wdac sdk on the server and each machine that you connect to.

$credential = Get-Credential
foreach ($server in @("ESS027521","ESS026412","ESS026488","ESS026191")) {
    $session = New-CimSession -ComputerName $server -Credential $credential
    $odbc = Get-OdbcDsn  -CimSession $session
    $server + ":"
    $odbc        
}

EF lazy and Eager loading–caught out! (Putting lazy load back on)

I was a bit optimistic in my last post on EF.  Turing off Lazy Loading (removing virtual) on a attribute does NOT imply eager loading.  Documentation is unlcear but confirmed by internet -

“IMPORTANT: You could easily think that, once you disable Lazy Loading, the framework will auto-load each and every related property: it won’t.” - http://www.ryadel.com/en/enable-or-disable-lazyloading-in-entity-framework/

So I’m back to the .Include on each of my get methods to ensure consistency.

Not sure the argument “Don’t worry, it’s a good thing! You don’t want your DB to be automatically wasted on each Entity query request.” isn’t a bit of a cop out.  I want to decide which of my attributes are composite – think car and wheels – and load those all the time.

Of course – I can do this with the .Include but its a bit less explicit.  Making the attribute virtual again will at least means the serialisation falls over if I forgot to include the Lazy Load/Serialisation fails as it’s outside the context – so that at least enforces my aggregate.

Hype driven development

Following on from Hype Driven Development (HDD) often implemented in CV++ – here is some more data, frameworks and api’s to consider:

-----Original Message-----

Subject: Amusing and/or point-making links for your arsenal!

http://foaas.com/

http://dayssincelastjavascriptframework.com/

http://www.ismycodeshit.com/

http://shouldiblamecaching.com/

http://vanilla-js.com/

http://shouldiuseacarousel.com/

On EF lazy loading and UML, EF and Serialisation

 

On EF and lazy loading and UML

A good few years ago (back in 2010) I looked at EF and one of my concerns was that you could not control the loading.  A had an object which had a collection of objects – like a car and wheels, and when I fetched the car the framework would not fetch the wheels at the same time – it went and got them one at a time.

This did not appear to be a scalable solution.

This time round things seem a lot better and I have been able to use the techniques here - https://msdn.microsoft.com/en-us/library/jj574232(v=vs.113).aspx – to control what gets loaded when.  Seems pretty neat.

I am fairly impressed with this as it allows me to decide what is always “Eagerly” loaded in the class model – what in OO days would have been an composite in my UML model and in the LINQ stuff optionally load the association.

On EF, Normalisation and Serialisation

I am using the EF (code first) objects as my “Business Objects”.  Not sure this has lead to massive time savings over raw ADO.NET as all that annotation is quite a pain.  Yes I could just generate the code (I started like that) but I use some pretty old school naming conventions on my database – Hungarian notation – because I’m old – so attribute names become compromised.

One of my other concerns with EF has been security but I’m assured that LINQ for EF will generate parameterised queries which are good (for security) and meet OWSA

So far I’m fairly happy that my object layer is not too normalised – but I’ve yet to tackle anything too serious – like inheritance and a class comprised of two tables.  Not sure about the former yet but a view will probably help the later.

A couple of things – if you are serialising (to JSON in my case for the service layer) then you need to decide how much of the tree you are bringing back and what you will serialise.

  [Table("tblEvent")]
    [DataContract]
    public partial class Event
    {
        [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
        public Event()
        {
            tblEventParticipants = new HashSet<tblEventParticipant>();
        }

        [Key]
        [Column("intEventId")]
        [DataMember]
        public int EventId { get; set; }

        [Column("dtePlannedDate")]
        [DataMember()]
        public DateTime PlannedDate { get; set; }

       [Column("dteActualDate ")]
       [DataMember()]
        public DateTime? ActualDate { get; set; }

        [Column("intOrganiserId")]
        [ForeignKey("Organiser")]
        [DataMember()]
        public int? OrganiserId { get; set; }

        [Column("intRouteId")]
        [DataMember()]
        [ForeignKey("Route")]
        public int RouteId { get; set; }

        [DataMember()]
        public tblOrganiser Organiser { get; set; }

        [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
        public virtual ICollection<tblEventParticipant> tblEventParticipants { get; set; }

       
        [DataMember()]
        public Route Route { get; set; }
    }

So I’m marking my classes with the  DataContract attribute, marking the members to be serialised with DataMember attribute and making sure they are always loaded by removing the virtual keyword (Eagerly loading) else the serialisation will fail.

One hurdle I’ve yet to cross is that I can see times when I may wish to load and serialise some of the associations – will need to work this through.

On very small tasks

Yes I am still going – and starting to get things done.  Spare time projects are hard to work on because code requires lots of long focus.  But, I’ve found that by really breaking down tasks (I believe they call that pebbleisation) to items taking less than an hour I can quickly pick things up and get something done.  On a full time project you can get the whole thing straight in your head and then go ahead and implement but when there’s days or even weeks between then you need to have really recorded what you were going to do.

 

Moveing classic resources (within subscription)

Well – according to the documentation then you can use the portal to move classic resource items around your groups.

“

• Virtual machines (classic) must be moved with the cloud service.
• Cloud service can only be moved when the move includes all its virtual machines.”

…

“To move classic resources to a new resource group within the same subscription, use the standard move operations through the portal, Azure PowerShell, Azure CLI, or REST API. You use the same operations as you use for moving Resource Manager resources.”

It even has a nice little picture showing it being done

I had a group with two resources – a VM and its cloud service – but the portal would not allow them to be moved.

So this nice little script does it

# prompt for credentials etc.
Login-AzureRmAccount
# select the correct subscription
Get-AzureRmSubscription -SubscriptionName "Stiona Software General" | Select-AzureRmSubscription
# get the ResourceID property of the two resources - luckily they have the same name in my case
$resources = Get-AzureRmResource -ResourceName fusionreports -ResourceGroupName fusionreports | Select -ExpandProperty ResourceId
# issue a move to the new group
Move-AzureRmResource -DestinationResourceGroupName fusionazure -ResourceId $resources

The $resources this -

/subscriptions/b9fee249-e903-4c4a-ade7-42982be9a20f/resourceGroups/fusionreports/providers/Microsoft.ClassicCompute/domainNa
mes/fusionreports
/subscriptions/b9fee249-e903-4c4a-ade7-42982be9a20f/resourceGroups/fusionreports/providers/Microsoft.ClassicCompute/virtualM
achines/fusionreports

After promt to move the old resource group was empty and the new one contained the vm and cloud service.

Nice hint from stack overflow on the use of  ExpandProperty parameter of Select-Object for getting an array of values from an array of objects.

Man – working with Azure classic sure is tough.