Merge pull request #134 from joolfe/feature/replace-var

Feature/replace var

CHANGELOG.md

@@ -1,3 +1,21 @@
+## [1.16.0](https://github.com/joolfe/postman-to-openapi/compare/1.15.0...1.16.0) (2021-08-01)
+
+
+### Features
+
+* replace variables from postman collection or new option ([75f8df1](https://github.com/joolfe/postman-to-openapi/commit/75f8df109bf6433c1933d08ebd1b11bab77a7e26))
+* var replacer first implementation ([768e59f](https://github.com/joolfe/postman-to-openapi/commit/768e59fc4a62a9d98ef394ca1ec3319e908b6531))
+
+
+### Documentation
+
+* update documentation with new replacement feature ([9cfd142](https://github.com/joolfe/postman-to-openapi/commit/9cfd1427d1d4735f8470077ca1a3a84b2a53c229))
+
+
+### Build System
+
+* update version for new feature ([32cdb47](https://github.com/joolfe/postman-to-openapi/commit/32cdb47b58289f453abb30407f3e6ed27384cf0b))
+
 ## [1.15.0](https://github.com/joolfe/postman-to-openapi/compare/1.14.0...1.15.0) (2021-07-30)
 
 

docs/index.md

@@ -18,7 +18,8 @@
 
 * Postman Collection v2.1 and v2.0.
 * OpenApi 3.0
-* 🆕 Cli available
+* Cli available
+* 🆕 Postman variables automatically replaced.
 * Basic info API from Postman info or customizable.
 * Basic method conversion (GET, POST, PUT...).
 * Support Postman folders as tags.
@@ -31,7 +32,7 @@
 * Provide meta-information as a markdown table.
 * Path depth configuration.
 * API Response parse from postman examples and from test code (status code).
-* [x-logo](https://github.com/Redocly/redoc/blob/master/docs/redoc-vendor-extensions.md#x-logo) extension support
+* [x-logo](https://github.com/Redocly/redoc/blob/master/docs/redoc-vendor-extensions.md#x-logo) extension support.
 
 See [Features](#features) section for more details about how to use each of this features.
 
@@ -129,6 +130,8 @@ The third parameter used in the library method is an `options` object containing
 | [externalDocs](#externaldocs-object) | Info about the API external documentation. |
 | [folders](#folders-object) | Config object for folders and nested folders in postman collection. |
 | [responseHeaders](#responseheaders-boolean) | Indicate if should parse the response headers from the collection examples. |
+| [replaceVars](#replacevars-boolean) | Boolean value to indicate if postman variables should be replaced.|
+| [additionalVars](#additionalvars-object) |  Object to provide additional values for variables replacement.|
 
 ### info (Object)
 
@@ -316,6 +319,33 @@ This flag indicates if the headers that are saved as part of the postman collect
 
 The default value is `true`, so headers are by default added to the response definition.
 
+### replaceVars (Boolean)
+
+This flag indicates if the [postman variables](https://learning.postman.com/docs/sending-requests/variables/) referenced in the postman collection should be replaced before generate the OpenAPI specs.
+
+If set to `true` all variable references contained in the postman collection as "{{variable}}" will be replaced by his value defined at [postman collection level](https://learning.postman.com/docs/sending-requests/variables/#defining-collection-variables) or values provided the [additionalVars Object](#additionalvars-object).
+
+Be aware that path variables defined as postman variables "{{variable}}" as for example a path like `https://api.io/users/{{user_id}}` will be also replaced if there exist a variable definition at postman collection or in the [additionalVars Object](#additionalvars-object).
+
+The default value for this flag is `false` as variable replacement has a performance cost.
+
+### additionalVars (Object)
+
+In postman, variables can be defined at different [scopes level](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) but only the ones defined at [postman collection level](https://learning.postman.com/docs/sending-requests/variables/#defining-collection-variables) will be saved inside collection file, to provide additional variable values, what we can call Global or Environment variables, there exist the `additionalVars` parameter.
+
+This parameter is a json Object that contain as key the variable name and as value the variable value, as for example:
+
+```js
+{
+    additionalVars: {
+        service : 'myService',
+        company : 'myCompany'
+    }
+}
+```
+
+Take into account that variable values provided in the `additionalVars` Object supersede those defined at Postman collection level.
+
 # Features
 
 ## Basic conversion
@@ -352,7 +382,7 @@ This library automatically transform query and headers parameters from Postman o
 
 The default schema used for parameters is `string` but the library try to infer the type of the parameters based on the value using regular expressions, the detected types are `integer`, `number`, `boolean` and `string`, if you find any problem in the inference process please open an issue.
 
-Path parameters are also automatically detected, this library look for [Postman variables](https://learning.postman.com/docs/sending-requests/variables/) in the url as `{{{variable}}}` and transform to a single curly brace expression as `{variable}` as supported by OpenAPI, also create the parameter definition using the variable name. To provide additional information about a path parameter you can [Pass Meta-information as markdown](#pass-meta-information-as-markdown).
+Path parameters are also automatically detected, this library look for [Postman variables](https://learning.postman.com/docs/sending-requests/variables/) in the url as "{{variable}}" and transform to a single curly brace expression as `{variable}` as supported by OpenAPI, also create the parameter definition using the variable name. To provide additional information about a path parameter you can [Pass Meta-information as markdown](#pass-meta-information-as-markdown). Be aware that if you use the `replaceVar` option the path parameter using Postman variables can be affected. See [replaceVars option](#replacevars-boolean)
 
 For headers and query fields you can indicate that this parameter is mandatory/required adding into the description the literal `[required]`. The library use a case insensitive regexp so all variations are supported (`[REQUIRED]`, `[Required]`...) and never mind the location inside the description (at the beginning, at the end...).
 

lib/index.js

@@ -4,14 +4,18 @@ const { promises: { writeFile, readFile } } = require('fs')
 const { dump } = require('js-yaml')
 const { parseMdTable } = require('./md-utils')
 const { version } = require('../package.json')
+const replacePostmanVariables = require('./var-replacer')
 
 async function postmanToOpenApi (input, output, {
   info = {}, defaultTag = 'default', pathDepth = 0,
   auth: optsAuth, servers, externalDocs = {}, folders = {},
-  responseHeaders = true
+  responseHeaders = true, replaceVars = false, additionalVars = {}
 } = {}) {
   // TODO validate?
-  const collectionFile = await readFile(input)
+  let collectionFile = await readFile(input, 'utf8')
+  if (replaceVars) {
+    collectionFile = replacePostmanVariables(collectionFile, additionalVars)
+  }
   const postmanJson = JSON.parse(collectionFile)
   const { item: items, variable = [] } = postmanJson
   const paths = {}
@@ -203,7 +207,7 @@ function mapFormData () {
 }
 
 /* Parse the Postman query and header and transform into OpenApi parameters */
-function parseParameters (query = [], header, paths, paramsMeta = {}) {
+function parseParameters (query, header, paths, paramsMeta = {}) {
   // parse Headers
   let parameters = header.reduce(mapParameters('header'), [])
   // parse Query
@@ -326,7 +330,7 @@ function parseOptsAuth (optAuth) {
 }
 
 /* From the path array compose the real path for OpenApi specs */
-function calculatePath (paths = [], pathDepth) {
+function calculatePath (paths, pathDepth) {
   paths = paths.slice(pathDepth) // path depth
   // replace repeated '{' and '}' chars
   return '/' + paths.map(path => path.replace(/([{}])\1+/g, '$1'))
@@ -337,24 +341,53 @@ function calculateDomains (protocol, hosts, port) {
   return protocol + '://' + hosts.join('.') + (port ? `:${port}` : '')
 }
 
-/** Support for collection V2 */
+/**
+ * To support postman collection v2 and variable replace we should parse the `url` or `url.raw` data
+ * without trust in the object as in v2 could not exist and if replaceVars = true then values cannot
+ * be correctly parsed
+ * @param {Object | String} url
+ * @returns a url structure as in postman v2.1 collections
+ */
 function scrapeURL (url) {
+  // Avoid parse empty url request
   if (url === '' || url.raw === '') {
     return { valid: false }
   }
-  if (typeof url === 'string' || url instanceof String) {
-    const objUrl = new URL(url)
-    return {
-      raw: url,
-      path: decodeURIComponent(objUrl.pathname).slice(1).split('/'),
-      query: [],
-      protocol: objUrl.protocol.slice(0, -1),
-      host: decodeURIComponent(objUrl.hostname).split('.'),
-      port: objUrl.port,
-      valid: true
-    }
+  const rawUrl = (typeof url === 'string' || url instanceof String) ? url : url.raw
+  const objUrl = new URL(rawUrl)
+  return {
+    raw: rawUrl,
+    path: decodeURIComponent(objUrl.pathname).slice(1).split('/'),
+    query: compoundQueryParams(objUrl.searchParams, url.query),
+    protocol: objUrl.protocol.slice(0, -1),
+    host: decodeURIComponent(objUrl.hostname).split('.'),
+    port: objUrl.port,
+    valid: true
   }
-  return { ...url, valid: true }
+}
+
+/**
+ * Calculate query parameters as postman collection
+ * @param {*} searchParams The searchParam instance from an URL object
+ * @param {*} queryCollection The postman collection query section
+ * @returns A query params array as created by postman collections Array(Obj)
+ */
+function compoundQueryParams (searchParams, queryCollection = []) {
+  // Prepare desc in query collection for easy search
+  const descMap = queryCollection.reduce((agr, { key, description }) => {
+    agr[key] = description
+    return agr
+  }, {})
+  // Create the query array of objects
+  const query = []
+  searchParams.forEach((value, key) => {
+    query.push({
+      key,
+      value,
+      ...(descMap[key] != null ? { description: descMap[key] } : {})
+    })
+  })
+  return query
 }
 
 /* Parse domains from operations or options */

lib/var-replacer.js

@@ -0,0 +1,23 @@
+const Mustache = require('mustache')
+
+/**
+ * Rewrite escapedValue() function to not delete undefined variables
+ */
+Mustache.Writer.prototype.escapedValue = function escapedValue (token, context, config) {
+  const value = context.lookup(token[1]) || `{{${token[1]}}}`
+  return String(value)
+}
+
+function replacePostmanVariables (collectionString, additionalVars = {}) {
+  const postmanJson = JSON.parse(collectionString)
+  const { variable } = postmanJson
+  const formatVars = variable.reduce((obj, { key, value }) => {
+    obj[key] = value
+    return obj
+  }, {})
+  // Merge collection vars with additional vars
+  const context = { ...formatVars, ...additionalVars }
+  return Mustache.render(collectionString, context)
+}
+
+module.exports = replacePostmanVariables

package.json

@@ -1,6 +1,6 @@
 {
   "name": "postman-to-openapi",
-  "version": "1.15.0",
+  "version": "1.16.0",
   "description": "Convert postman collection to OpenAPI spec",
   "main": "lib/index.js",
   "types": "types/index.d.ts",
@@ -88,7 +88,8 @@
   "dependencies": {
     "commander": "^7.2.0",
     "js-yaml": "^4.1.0",
-    "marked": "^2.0.7"
+    "marked": "^2.0.7",
+    "mustache": "^4.2.0"
   },
   "husky": {
     "hooks": {

test/index.spec.js

@@ -50,6 +50,8 @@ const EXPECTED_AUTH_REQUEST = readFileSync('./test/resources/output/AuthRequest.
 const EXPECTED_RESPONSES_NO_HEADERS = readFileSync('./test/resources/output/ResponsesNoHeaders.yml', 'utf8')
 const EXPECTED_FORM_DATA = readFileSync('./test/resources/output/FormData.yml', 'utf8')
 const EXPECTED_FORM_URLENCODED = readFileSync('./test/resources/output/FormUrlencoded.yml', 'utf8')
+const EXPECTED_VARIABLES = readFileSync('./test/resources/output/Variables.yml', 'utf8')
+const EXPECTED_VARIABLES_ADDITIONAL = readFileSync('./test/resources/output/VariablesAdditional.yml', 'utf8')
 
 const AUTH_DEFINITIONS = {
   myCustomAuth: {
@@ -106,6 +108,7 @@ describe('Library specs', function () {
       const COLLECTION_AUTH_REQUEST = `./test/resources/input/${version}/AuthRequest.json`
       const COLLECTION_FORM_DATA = `./test/resources/input/${version}/FormData.json`
       const COLLECTION_FORM_URLENCODED = `./test/resources/input/${version}/FormUrlencoded.json`
+      const COLLECTION_VARIABLES = `./test/resources/input/${version}/Variables.json`
 
       it('should work with a basic transform', async function () {
         const result = await postmanToOpenApi(COLLECTION_BASIC, OUTPUT_PATH, {})
@@ -409,6 +412,22 @@ describe('Library specs', function () {
         const result = await postmanToOpenApi(COLLECTION_FORM_URLENCODED, OUTPUT_PATH, {})
         equal(result, EXPECTED_FORM_URLENCODED)
       })
+
+      it('should replace postman variables if feature activated', async function () {
+        const result = await postmanToOpenApi(COLLECTION_VARIABLES, OUTPUT_PATH, { replaceVars: true })
+        equal(result, EXPECTED_VARIABLES)
+      })
+
+      it('should use additional variables for replace', async function () {
+        const result = await postmanToOpenApi(COLLECTION_VARIABLES, OUTPUT_PATH, {
+          replaceVars: true,
+          additionalVars: {
+            company: 'myCompany',
+            service: 'myService'
+          }
+        })
+        equal(result, EXPECTED_VARIABLES_ADDITIONAL)
+      })
     })
   })