Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: weo-edu/express-robots
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: mixmaxhq/express-robots
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Able to merge. These branches can be automatically merged.
  • 18 commits
  • 9 files changed
  • 8 contributors

Commits on Nov 19, 2018

  1. Remove github as-array dependency

    Eli Skeggs committed Nov 19, 2018

    Unverified

    No user is associated with the committer email.
    Copy the full SHA
    e00da34 View commit details
  2. Remove all dependencies

    Eli Skeggs committed Nov 19, 2018

    Unverified

    No user is associated with the committer email.
    Copy the full SHA
    48cdc49 View commit details
  3. Merge pull request #1 from mixmaxhq/remove-as-array

    Remove github as-array dependency
    skeggse authored Nov 19, 2018

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    1097219 View commit details

Commits on Dec 12, 2019

  1. ci: use jenkins

    Eli Skeggs committed Dec 12, 2019

    Unverified

    No user is associated with the committer email.
    Copy the full SHA
    10ed203 View commit details
  2. ci: add semantic-release, enforce commitlint

    Eli Skeggs committed Dec 12, 2019

    Unverified

    No user is associated with the committer email.
    Copy the full SHA
    344e9a9 View commit details
  3. chore: add ci script to package

    Eli Skeggs committed Dec 12, 2019

    Unverified

    No user is associated with the committer email.
    Copy the full SHA
    7a0e6bb View commit details
  4. Merge pull request #3 from mixmaxhq/add-semantic-release

    add semantic release, use jenkins
    skeggse authored Dec 12, 2019

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    3b5ce25 View commit details
  5. fix: use express router instead of application

    Previously, this package created an express application, and attached the /robots.txt route to that
    application. Generally, this isn't a useful feature for real application, and it can cause
    x-powered-by to be visible on (all) response requests. Instead of using an application, switch the
    package to use a router.
    
    BREAKING CHANGE: The main export now returns an express router instead of an express application.
    skeggse authored and Eli Skeggs committed Dec 12, 2019

    Unverified

    No user is associated with the committer email.
    Copy the full SHA
    b07fa43 View commit details
  6. Merge pull request #2 from mixmaxhq/eli/use-router

    fix: use express router instead of application
    skeggse authored Dec 12, 2019

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    06e1306 View commit details
  7. chore(release): 1.0.0 [skip ci]

    ## [1.0.0](mixmaxhq/express-robots@v0.1.6...v1.0.0) (2019-12-12)
    
    ### ⚠ BREAKING CHANGES
    
    * The main export now returns an express router instead of an express application.
    
    ### Bug Fixes
    
    * use express router instead of application ([b07fa43](mixmaxhq@b07fa43))
    semantic-release-bot committed Dec 12, 2019
    Copy the full SHA
    987ca7c View commit details

Commits on May 17, 2020

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    b250696 View commit details

Commits on May 20, 2020

  1. Merge pull request #5 from jeemok/patch-1

    Corrected the output `User-agent`
    bradvogel authored May 20, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    efb81a6 View commit details

Commits on Jul 21, 2023

  1. chore: add PR template

    guilhermemj committed Jul 21, 2023
    Copy the full SHA
    dd75a3c View commit details
  2. Copy the full SHA
    aef77b0 View commit details
  3. chore: remove jenkinsfile

    guilhermemj committed Jul 21, 2023
    Copy the full SHA
    b48848b View commit details
  4. Merge pull request #6 from mixmaxhq/guilherme/repo-chores

    chore: update pr template and ci configuration
    guilhermemj authored Jul 21, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    89adfff View commit details

Commits on Aug 22, 2024

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    2f2a136 View commit details

Commits on Aug 23, 2024

  1. Merge pull request #7 from mixmaxhq/raphaelbs-patch-1

    fix: reassigning to const variable
    raphaelbs authored Aug 23, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    0001304 View commit details
Showing with 9,594 additions and 41 deletions.
  1. +65 −0 .github/pull_request_template.md
  2. +13 −0 .github/workflows/ci.yml
  3. +1 −0 .nvmrc
  4. +10 −0 CHANGELOG.md
  5. +9 −9 README.md
  6. +6 −11 index.js
  7. +9,434 −0 package-lock.json
  8. +36 −12 package.json
  9. +20 −9 test/index.js
65 changes: 65 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
<!-- Author/Reviewer expectations are described in the last comment -->

> **Note** - Since this is a public repository, make sure that we're not publishing private data in the code, commit comments, or this PR.
## 📚 Context/Description Behind The Change
<!--
What changes have you made to the code, and why?
As skilled engineers, we all know how to read and interpret code, so take this opportunity
to be a bit more verbal about your changes.
Giving the context behind your changes will make your PR review quicker,
as the reviewer will not need to guess your intent.
-->

## 🚨 Potential Risks & What To Monitor After Deployment
<!--
What can go wrong with this deploy?
Does it touch any critical services?
How will these changes affect adjacent code/features?
How will we handle any adverse issues?
-->

## 🧑‍🔬 How Has This Been Tested?
<!--
Imagine: How do I (and the Reviewer):
How do we know this PR does what it's supposed to do?
How do we ensure that adjacent code/features are still working?
How do we evaluate the performance implications of this PR?
-->

## 🚚 Release Plan
<!--
Imagine: If you had to leave in a rush, what should the backup engineer do to deploy this?
Add any tasks that need to be done before/during/after release (i.e, creating indices, deploying
other services, bumping modules, notifying PM or other engineers, etc).
-->



<!--
## 🤝 Expectations
When Opening/Reviewing a PR, please keep in mind:
### As the PR Author
- Provide all the necessary context on "Why" you have performed your changes
- Assume that the reviewer has no previous knowledge about the codebase: would he/she be able to
review it the way it is right now?
- If changes are extensive, break them into smaller commits that tell a story, instead of 1 commit
with 15 files changed.
- Split Refactors and New-Code-Changes into different commits, ideally: different PRs.
- Test your code before requesting review — unless some rare exceptions, we shouldn't ship any code
without testing it before.
- If your PR is UI related, consider adding screenshots/videos with the behavior and before/after.
### As the PR Reviewer
- Be kind. Don't nitpick.
- Expect to have all the necessary context to review the PR on (or linked on) the PR itself.
- When in doubt: ask.
- Validate if the author's tests have any missing coverage points. Do not approve an untested PR.
- Pay extra attention to the "Potential Risks" and "Release Plan" sections.
- If the PR alters the product UI, consider checking out the branch to visually inspect it.
-->
13 changes: 13 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: continuous-integration

on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
checks:
uses: mixmaxhq/github-workflows-public/.github/workflows/checks.yml@main
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
12.22.6
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## [1.0.0](https://github.com/mixmaxhq/express-robots/compare/v0.1.6...v1.0.0) (2019-12-12)


### ⚠ BREAKING CHANGES

* The main export now returns an express router instead of an express application.

### Bug Fixes

* use express router instead of application ([b07fa43](https://github.com/mixmaxhq/express-robots/commit/b07fa43d31f553dd6b1fbcd9b7f6f372cf5515d7))
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -13,25 +13,25 @@ app.use(robots(__dirname + '/robots.txt'));
### Basic object

```javascript
app.use(robots({UserAgent: '*', Disallow: '/'}))
app.use(robots({ UserAgent: '*', Disallow: '/' }))
```

#### Will produce:
```
UserAgent: *
User-agent: *
Disallow: /
```

### Crawl Delay
You can optionally pass a CrawlDelay in just like passing in Disallow

```javascript
app.use(robots({UserAgent: '*', Disallow: '/', CrawlDelay: '5'}))
app.use(robots({ UserAgent: '*', Disallow: '/', CrawlDelay: '5' }))
```

#### Will produce:
```
UserAgent: *
User-agent: *
Disallow: /
Crawl-delay: 5
```
@@ -53,9 +53,9 @@ app.use(robots([

#### Will produce:
```
UserAgent: Googlebot
User-agent: Googlebot
Disallow: /no-google
UserAgent: Bingbot
User-agent: Bingbot
Disallow: /no-bing
```

@@ -76,11 +76,11 @@ app.use(robots([

#### Will produce:
```
UserAgent: Googlebot
UserAgent: Slurp
User-agent: Googlebot
User-agent: Slurp
Disallow: /no-google
Disallow: /no-yahoo
UserAgent: *
User-agent: *
Disallow: /no-bots
Disallow: /still-no-bots
```
17 changes: 6 additions & 11 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,14 +1,9 @@
var fs = require('fs');
var asArray = require('as-array');
const fs = require('fs');

if (!Array.isArray) {
Array.isArray = function(arg) {
return Object.prototype.toString.call(arg) === '[object Array]';
};
}
const asArray = (value = []) => Array.isArray(value) ? value : [value];

module.exports = function(robots) {
var app = require('express')();
module.exports = function(robots, {express = require('express')} = {}) {
const app = express.Router();

if(robots) {
robots = 'string' === typeof robots
@@ -27,7 +22,7 @@ module.exports = function(robots) {

function render(robots) {
return asArray(robots).map(function(robot) {
var userAgentArray = [];
let userAgentArray = [];
if (Array.isArray(robot.UserAgent)) {
userAgentArray = robot.UserAgent.map(function(userAgent) {
return 'User-agent: ' + userAgent
@@ -47,4 +42,4 @@ function render(robots) {
return 'Disallow: ' + disallow;
})).join('\n');
}).join('\n');
}
}
Loading