Restore markdownlint tests (#12549)

Co-authored-by: Travis Plunk <travis.plunk@microsoft.com>

.github/ISSUE_TEMPLATE/Bug_Report.md

@@ -20,25 +20,25 @@ This repository is **ONLY** for PowerShell Core 6 and PowerShell 7+ issues.
 
 -->
 
-# Steps to reproduce
+## Steps to reproduce
 
 ```powershell
 
 ```
 
-# Expected behavior
+## Expected behavior
 
 ```none
 
 ```
 
-# Actual behavior
+## Actual behavior
 
 ```none
 
 ```
 
-# Environment data
+## Environment data
 
 <!-- provide the output of $PSVersionTable -->
 

.github/ISSUE_TEMPLATE/Distribution_Request.md

@@ -7,24 +7,24 @@ assignees: ''
 
 ---
 
-# Details of the Distribution
+## Details of the Distribution
 
-- Name of the Distribution: 
+- Name of the Distribution:
 - Version of the Distribution:
 - Package Types
-  - [ ] Deb
-  - [ ] RPM
-  - [ ] Tar.gz
-  - Snap - Please file issue in https://github.com/powershell/powershell-snap.  This issues type is unrelated to snap packages with a distribution neutral.
-- Processor Architecture (One per request): 
+    - [ ] Deb
+    - [ ] RPM
+    - [ ] Tar.gz
+    - Snap - Please file issue in https://github.com/powershell/powershell-snap.  This issues type is unrelated to snap packages with a distribution neutral.
+- Processor Architecture (One per request):
 - [ ] **Required** - An issues has been filed to create a Docker image in https://github.com/powershell/powershell-docker
 - The following is a requirement for supporting a distribution **without exception.**
-  - [ ] The version and architecture of the Distribution is [supported by .NET Core](https://github.com/dotnet/core/blob/master/release-notes/3.0/3.0-supported-os.md#linux).
+    - [ ] The version and architecture of the Distribution is [supported by .NET Core](https://github.com/dotnet/core/blob/master/release-notes/3.0/3.0-supported-os.md#linux).
 - The following are requirements for supporting a distribution.
-  Please write a justification for any exception where these criteria are not met and 
+  Please write a justification for any exception where these criteria are not met and
   the PowerShell committee will review the request.
-  - [ ] The version of the Distribution is supported for at least one year.
-  - [ ] The version of the Distribution is not an [interim release](https://ubuntu.com/about/release-cycle) or equivalent.
+    - [ ] The version of the Distribution is supported for at least one year.
+    - [ ] The version of the Distribution is not an [interim release](https://ubuntu.com/about/release-cycle) or equivalent.
 
 ## Progress - For PowerShell Team **ONLY**
 

.github/ISSUE_TEMPLATE/Feature_Request.md

@@ -7,7 +7,7 @@ assignees: ''
 
 ---
 
-# Summary of the new feature/enhancement
+## Summary of the new feature/enhancement
 
 <!-- 
 A clear and concise description of what the problem is that the new feature would solve.
@@ -15,7 +15,7 @@ Try formulating it in user story style (if applicable):
 'As a user I want X so that Y.' with X being the being the action and Y being the value of the action.
 -->
 
-# Proposed technical implementation details (optional)
+## Proposed technical implementation details (optional)
 
 <!-- 
 A clear and concise description of what you want to happen.

.github/ISSUE_TEMPLATE/Release_Process.md

@@ -12,7 +12,7 @@ This template is for maintainers to create an issues to track the release proces
 Please **only** use this template if you are a maintainer.
 -->
 
-# Release Process for v6.x.x
+## Checklist
 
 - [ ] Verify that `PowerShell-Native` has been updated/released as needed.
 - [ ] Check for `PowerShellGet` and `PackageManagement` release plans.

.github/ISSUE_TEMPLATE/Security_Issue_Report.md

@@ -7,7 +7,7 @@ assignees: 'TravisEz13'
 
 ---
 
-# Security Issue
+## Security Issue
 
 Excerpt from [Issue Management - Security Vulnerabilities](https://github.com/PowerShell/PowerShell/blob/master/.github/SECURITY.md)
 

.vsts-ci/misc-analysis.yml

@@ -68,3 +68,8 @@ jobs:
       }
     displayName: Run Common Tests
     condition: succeededOrFailed()
+  - task: ms.vss-governance-buildtask.governance-build-task-component-detection.ComponentGovernanceComponentDetection@0
+    displayName: 'Component Detection'
+    inputs:
+      sourceScanPath: '$(Build.SourcesDirectory)'
+      snapshotForceEnabled: true

README.md

@@ -144,6 +144,7 @@ We have a Gitter Room which you can join below.
 [![Join the chat](https://img.shields.io/static/v1.svg?label=chat&message=on%20gitter&color=informational&logo=gitter)](https://gitter.im/PowerShell/PowerShell?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
 There is also the community driven PowerShell Virtual User Group, which you can join on:
+
 * [Slack](https://aka.ms/psslack)
 * [Discord](https://aka.ms/psdiscord)
 

test/common/markdown/gulpfile.js

@@ -0,0 +1,60 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+function runTest() {
+    "use strict";
+    var gulp = require("gulp");
+    var concat = require("gulp-concat");
+    var through2 = require("through2");
+    var markdownlint = require("markdownlint");
+
+    gulp.task("test-mdsyntax", function task() {
+        var paths = [];
+        var rootpath;
+
+        // assign --repoRoot <rootpath> into rootpath
+        var j = process.argv.indexOf("--rootpath");
+        if (j > -1) {
+            rootpath = process.argv[j + 1];
+        }
+
+        if (rootpath === null) {
+            throw "--rootpath <repoRoot> must be specified before all other parameters";
+        }
+
+        // parse --filter into paths.  --rootpath must be specified first.
+        j = process.argv.indexOf("--filter");
+        if (j > -1) {
+            var filters = process.argv[j + 1].split(",");
+            filters.forEach(function(filter) {
+                paths.push(rootpath + "/" + filter);
+            }, this);
+        }
+
+        if (paths.length === 0) {
+            throw "--filter <filter relative to repoRoot> must be specified";
+        }
+
+        var rootJsonFile = rootpath + "/.markdownlint.json";
+        var fs = require("fs");
+        fs.appendFileSync("markdownissues.txt", "--EMPTY--\r\n");
+        return gulp.src(paths, { "read": false })
+            .pipe(through2.obj(function obj(file, enc, next) {
+                markdownlint({
+                        "files": [file.path],
+                        "config": require(rootJsonFile)
+                    },
+                    function callback(err, result) {
+                        var resultString = (result || "").toString();
+                        if (resultString) {
+                            file.contents = Buffer.from(resultString);
+                        }
+                        next(err, file);
+                    });
+            }))
+            .pipe(concat("markdownissues.txt", { newLine: "\r\n" }))
+            .pipe(gulp.dest("."));
+    });
+}
+
+runTest();

test/common/markdown/markdown-link.tests.ps1

@@ -105,7 +105,7 @@ Describe "Verify Markdown Links" {
                         $prefix = $url.Substring(0,7)
 
                         # Logging for diagnosability.  Azure DevOps sometimes redacts the full url.
-                        Write-Verbose "prefix: '$prefix'" -Verbose
+                        Write-Verbose "prefix: '$prefix'"
                         if($url -match '^http(s)?:')
                         {
                             # If invoke-WebRequest can handle the URL, re-verify, with 6 retries

test/common/markdown/markdown.tests.ps1

@@ -0,0 +1,100 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+Import-Module HelpersCommon
+$moduleRootFilePath = Split-Path -Path $PSScriptRoot -Parent
+
+# Identify the repository root path of the resource module
+$repoRootPath = (Resolve-Path -LiteralPath (Join-Path $moduleRootFilePath "../..")).ProviderPath
+$repoRootPathFound = $false
+
+Describe 'Common Tests - Validate Markdown Files' -Tag 'CI' {
+    BeforeAll {
+        Push-Location $psscriptroot
+        $skip = $false
+        $NpmInstalled = "not installed"
+        if (Get-Command -Name 'yarn' -ErrorAction SilentlyContinue)
+        {
+            $NpmInstalled = "Installed"
+            Write-Verbose -Message "Checking if Gulp is installed. This may take a few moments." -Verbose
+            start-nativeExecution { yarn }
+            if(!(Get-Command -Name 'gulp' -ErrorAction SilentlyContinue))
+            {
+                start-nativeExecution {
+                    sudo yarn global add 'gulp@4.0.2'
+                }
+            }
+            if(!(Get-Command -Name 'node' -ErrorAction SilentlyContinue))
+            {
+                throw "node not found"
+            }
+        }
+        if(!(Get-Command -Name 'node' -ErrorAction SilentlyContinue))
+        {
+            <#
+                On Windows, pre-requisites are missing
+                For now we will skip, and write a warning.  Work to resolve this is tracked in:
+                https://github.com/PowerShell/PowerShell/issues/3429
+            #>
+            Write-Warning "Node and yarn are required to run this test"
+            $skip = $true
+        }
+
+        $mdIssuesPath = Join-Path -Path $PSScriptRoot -ChildPath "markdownissues.txt"
+        Remove-Item -Path $mdIssuesPath -Force -ErrorAction SilentlyContinue
+    }
+
+    AfterAll {
+        Pop-Location
+    }
+
+    It "Should not have errors in any markdown files" -Skip:$skip {
+        $NpmInstalled | Should -BeExactly "Installed"
+        $mdErrors = 0
+        Push-Location -Path $PSScriptRoot
+        try
+        {
+            $docsToTest = @(
+                './.github/*.md'
+                './README.md'
+                './demos/python/*.md'
+                './docker/*.md'
+                './docs/building/*.md'
+                './docs/community/*.md'
+                './docs/host-powershell/*.md'
+                './docs/cmdlet-example/*.md'
+                './docs/maintainers/*.md'
+                './test/powershell/README.md'
+                './tools/*.md'
+                './.github/ISSUE_TEMPLATE/*.md'
+            )
+            $filter = ($docsToTest -join ',')
+
+            # Gulp 4 beta is returning non-zero exit code even when there is not an error
+            Start-NativeExecution {
+                    &"gulp" test-mdsyntax --silent `
+                        --rootpath $repoRootPath `
+                        --filter $filter
+                } -VerboseOutputOnError -IgnoreExitcode
+
+        }
+        finally
+        {
+            Pop-Location
+        }
+
+        $mdIssuesPath | Should -Exist
+
+        [string[]] $markdownErrors = Get-Content -Path $mdIssuesPath
+        Remove-Item -Path $mdIssuesPath -Force -ErrorAction SilentlyContinue
+
+        if ($markdownErrors -ne "--EMPTY--")
+        {
+            $markdownErrors += ' (See https://github.com/DavidAnson/markdownlint/blob/master/doc/Rules.md for an explanation of the error codes)'
+        }
+
+        $markdownErrors | Write-Host
+
+        $markdownErrors -join "`n" | Should -BeExactly "--EMPTY--"
+    }
+}

test/common/markdown/package.json

@@ -0,0 +1,26 @@
+{
+    "name": "powershell.common.markdown.tests",
+    "private": true,
+    "version": "1.0.0",
+    "description": "The PowerShell Common Markdown Tests.",
+    "main": "gulpfile.js",
+    "dependencies": {
+        "gulp": "^4.0.2",
+        "markdownlint": "^0.20.2",
+        "through2": "^3.0.1"
+    },
+    "devDependencies": {
+        "gulp-concat": "^2.6.1",
+        "gulp-debug": "^4.0.0"
+    },
+    "repository": {
+        "type": "git",
+        "url": "git+https://github.com/PowerShell/PowerShell.git"
+    },
+    "author": "Microsoft Corporation",
+    "license": "MIT",
+    "bugs": {
+        "url": "https://github.com/PowerShell/PowerShell/issues"
+    },
+    "homepage": "https://github.com/PowerShell/PowerShell#readme"
+}

tools/install-powershell.ps1-README.md

@@ -0,0 +1,25 @@
+# install-powershell.ps1
+
+## Features of install-powershell.ps1
+
+* Can be called directly from git
+* Optionally allows install of the latest Preview build
+* Optionally allows install of the Daily build
+* Optionally installs using the latest MSI
+* Automatically looks up latest version via git tags
+* Optionally installs silently
+* Optionally adds the install location to Path environment variable
+
+## Examples
+
+### Install PowerShell Core Daily Build
+
+```PowerShell
+Invoke-Expression "& { $(Invoke-RestMethod 'https://aka.ms/install-powershell.ps1') } -daily"
+```
+
+### Install PowerShell Core using the MSI installer
+
+```PowerShell
+Invoke-Expression "& { $(Invoke-RestMethod 'https://aka.ms/install-powershell.ps1') } -UseMSI"
+```

tools/install-powershell.sh-README.md

@@ -66,29 +66,3 @@ bash <(wget -O - https://raw.githubusercontent.com/PowerShell/PowerShell/master/
 ### Installation To do list
 
 * Detect and wait when package manager is busy/locked? - at least Ubuntu (CentOS does this internally)
-
-# install-powershell.ps1
-
-## Features of install-powershell.ps1
-
-* Can be called directly from git
-* Optionally allows install of the latest Preview build
-* Optionally allows install of the Daily build
-* Optionally installs using the latest MSI 
-* Automatically looks up latest version via git tags
-* Optionally installs silently
-* Optionally adds the install location to Path environment variable
-
-## Examples
-
-### Install PowerShell Core Daily Build
-
-```PowerShell
-Invoke-Expression "& { $(Invoke-RestMethod 'https://aka.ms/install-powershell.ps1') } -daily"
-```
-### Install PowerShell Core using the MSI installer
-
-```PowerShell
-Invoke-Expression "& { $(Invoke-RestMethod 'https://aka.ms/install-powershell.ps1') } -UseMSI"
-```
-