Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose request verb, body, url, query params and headers in pre/post scripts. #194

Closed
hsanson opened this issue Aug 29, 2024 · 12 comments · Fixed by #252
Closed

Expose request verb, body, url, query params and headers in pre/post scripts. #194

hsanson opened this issue Aug 29, 2024 · 12 comments · Fixed by #252
Assignees
Labels
enhancement New feature or request

Comments

@hsanson
Copy link

hsanson commented Aug 29, 2024

In pre/post scripts the request seems to only have functions to set/get variables . I have some services that use a custom HMAC authentication that requires computation of a signature that is then sent in the Authorization header.

The signature is computed by generating a canonical string that contains the request verb, url, query params, body, and a timestamp that is then encrypted with a token. Details can be found in the API-AUTH ruby gem, that is what the service I use uses to implement authentication.

If the request object in pre/post scripts exposes this information it would be easy for me to implement the signature and add the header to the request before it is sent to the server.

@gorillamoe gorillamoe self-assigned this Aug 29, 2024
@gorillamoe gorillamoe added the enhancement New feature or request label Aug 29, 2024
@gorillamoe
Copy link
Member

Just a quick heads up. Development should start this weekend 🥵

@gorillamoe gorillamoe linked a pull request Sep 30, 2024 that will close this issue
@gorillamoe
Copy link
Member

gorillamoe commented Oct 1, 2024

@hsanson, with the current open PR this here is easily achievable:

Something like this is now easily possible:

./../scripts/pre-token-gen.js

// https://www.jetbrains.com/help/idea/http-response-reference.html#request-properties
const url = new URL(request.url.tryGetSubstituted());
const method = request.method;
const params = url.searchParams;
const unix_timestamp = Math.floor(Date.now() / 1000);
const token_raw = request.variables.get('TOKEN_RAW');
const key1 = params.get('key1');
const computed_token = `${method}${token_raw}${key1}${unix_timestamp}`;

// Either use request.variables.set which is only valid for the current request
// or use client.global.set("COMPUTED_TOKEN", computed_token) to store the value globally
// and persist it across restarts
request.variables.set('COMPUTED_TOKEN', computed_token);
const contentTypeHeader = request.headers.findByName('Content-Type');
if (contentTypeHeader) {
  client.log("Content-Type:" + contentTypeHeader.getRawValue());
}
client.log({ url, method, params });

advanced-scripting.http

< {%
request.variables.set('TOKEN_RAW', 'THIS_IS_A_TOKEN');
%}
< ./../scripts/pre-token-gen.js
POST https://httpbin.org/post?key1=value1 HTTP/1.1
Accept: application/json
Content-Type: application/json
Token: {{COMPUTED_TOKEN}}

{
  "foobar": "lul"
}

Will merge it today. Let me know if this solves your issue.

@gorillamoe
Copy link
Member

@hsanson
Copy link
Author

hsanson commented Oct 5, 2024

@gorillamoe thanks alot for implementing this feature. With this I can finally stop using Insomnia/Postman in my work flow. Unfortunatelly I must be doing something wrong since I am unable to get this to work.

I have this http request for login that after successful login it should set a session ID and Token in the global context:

# @name LOGIN
POST https://{{HOST}}/graphql HTTP/1.1
Content-Type: application/json
Accept: application/json
X-REQUEST-TYPE: GraphQL

mutation login($email: String!, $password: String!) {
  userSignIn(input: {email: $email, password: $password}) {
    secret
    user {
      email
      name
      id
      tenant {
        id
        name
      }
    }
    session {
      id
      access_id
    }
  }
}

{
  "email": "{{USER}}",
  "password": "{{PASS}}"
}

> {%
  client.global.set("SESSION_ID",
    response.body.json.data.userSignIn.session.id);
  client.global.set("SESSION_TOKEN",
    response.body.json.data.userSignIn.secret);
%}

This requests works fine and I get the response. After login I am trying to make another request (separate http file) to retrive some info that requires HMAC signature. I have many different request so I want to login once, and then use the retrieved session id and token to make other requests.

# @name TENANTS
< ./hmack-auth.js
POST https://{{HOST}}/graphql HTTP/1.1
Content-Type: application/json
Accept: application/json
X-REQUEST-TYPE: GraphQL
Authorization: APIAuth {{SESSION_ID}}:{{SIGNATURE}}
X-Date: {{XDate}}
Content-Md5: {{MD5Hash}}

query getTenant {
  currentUser {
    id
    tenant {
      name
      id
    }
  }
}

This is the script:

const MD5      = require('crypto-js/md5');
const Hex      = require('crypto-js/enc-hex');
const Base64   = require('crypto-js/enc-base64');
const hmacSHA1 = require('crypto-js/hmac-sha1');
const hmac256  = require('crypto-js/hmac-sha256');
const sha256   = require("crypto-js/sha256");

// Session ID and Token are set by `login.http`
const sess_id = client.global.get("SESSION_ID");
const sess_token = client.global.get("SESSION_TOKEN");
const url = new URL(request.url.tryGetSubstituted());
const method = request.method;
const path = url.path;
const typeHeader = request.headers.findByName('Content-Type')
const type = typeHeader.getRawValue()
const body = request.body.tryGetSubstituted();

const xDate = (new Date()).toUTCString();
const md5Hash = Base64.stringify(MD5(body));
const canonicalStr = [method, type, md5Hash, path, xDate].join(',');
const signature = Hex.stringify(hmacSHA1(canonicalStr, secret));

client.log("Session ${SESSION_ID}")
client.log("Token ${SESSION_TOKEN}")
client.log("Signature ${SIGNATURE}")
request.variables.set("SESSION_ID", sess_id);
request.variables.set("SIGNATURE", signature);
request.variables.set("XDATE", xDate);
request.variables.set("MD5HASH", md5Hash);

Running http request I get warnings that none of the variables SIGNATURE, SESSION_ID, XDATE, etc are not set.

Questions:

  • I have zero visibility of what is going behind the scenes which makes debugging this difficult. Setting debug = true and enabling script_output in winbar do not seem to do anything. Where is the script output supposed to be available?
  • I am not sure if the pre-script is being executed. There are no errors or any output shown so not sure what is going on.
  • If there is a command like the toggle one that show script output instead of headers would be nice.
  • Also is there any way to see list of variables defined at the global and request scopes? this would greatly help with visibility and facilitate debuging.

@gorillamoe gorillamoe reopened this Oct 5, 2024
@gorillamoe
Copy link
Member

I'll check on that. Thanks 🙏🏾👍🏾 for the examples.

@gorillamoe
Copy link
Member

gorillamoe commented Oct 5, 2024

@hsanso, you should see all output err and stdout on :messages

But I think you catched a bug 🐛

@hsanson
Copy link
Author

hsanson commented Oct 5, 2024

@gorillamoe thanks for the tip. I can see now that the global variables were not being set:

 header: x-request-type value: graphql                                                                                                                              
"get_tenants.http" 19 lines --5%--                                                                                                                                 
"[Prompt]" [+] 1 line --100%--                                                                                                                                     
"login.http" 36 lines --52%--                                                                                                                                      
header: x-request-type value: graphql                                                                                                                              
/home/.local/share/nvim/lazy/kulala.nvim/lua/kulala/tmp/scripts/requests/5ecbb120-5c7a-43d2-aee2-c312d744a012.js:4                                          
    response.body.json.data.userSignIn.session.id);                                                                                                                
                       ^                                                                                                                                           
TypeError: Cannot read properties of undefined (reading 'data')                                                                                                    
    at Object.<anonymous> (/home/ryujin/.local/share/nvim/lazy/kulala.nvim/lua/kulala/tmp/scripts/requests/5ecbb120-5c7a-43d2-aee2-c312d744a012.js:4:24)           
    at Module._compile (node:internal/modules/cjs/loader:1469:14)                                                                                                  
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)                                                                                           
    at Module.load (node:internal/modules/cjs/loader:1288:32)                                                                                                      
    at Module._load (node:internal/modules/cjs/loader:1104:12)                                                                                                     
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)                                                                         
    at node:internal/main/run_main_module:28:49 

For some reason response.body.json is undefined. Also tried response.body.$.json with same results.

@gorillamoe
Copy link
Member

gorillamoe commented Oct 5, 2024

This here works for me on the current v4.0.4 release:

login.http

# @name LOGIN
POST https://httpbin.org/post
Content-Type: application/json
Accept: application/json

{
  "data": {
    "userSignIn": {
      "session": {
        "id": "1234567890",
        "secret": "supersecret"
      }
    }
  }
}

> {%
  client.global.set("SESSION_ID",
    response.body.json.data.userSignIn.session.id);
  client.global.set("SESSION_TOKEN",
    response.body.json.data.userSignIn.secret);
%}

hmac-auth.js

const MD5      = require('crypto-js/md5');
const Hex      = require('crypto-js/enc-hex');
const Base64   = require('crypto-js/enc-base64');
const hmacSHA1 = require('crypto-js/hmac-sha1');
const hmac256  = require('crypto-js/hmac-sha256');
const sha256   = require("crypto-js/sha256");

// Session ID and Token are set by `login.http`
const sess_id = client.global.get("SESSION_ID");
const sess_token = client.global.get("SESSION_TOKEN");
const url = new URL(request.url.tryGetSubstituted());
const method = request.method;
const path = url.path;
const typeHeader = request.headers.findByName('Content-Type')
client.log(typeHeader)
const type = typeHeader.getRawValue()
const body = request.body.tryGetSubstituted();

const xDate = (new Date()).toUTCString();
const md5Hash = Base64.stringify(MD5(body));
const canonicalStr = [method, type, md5Hash, path, xDate].join(',');
const secret = "foobar";
const signature = Hex.stringify(hmacSHA1(canonicalStr, secret));

client.log("Session", sess_id)
client.log("Token", sess_token)
client.log("Signature", signature)
request.variables.set("SESSION_ID", sess_id);
request.variables.set("SIGNATURE", signature);
request.variables.set("XDATE", xDate);
request.variables.set("MD5HASH", md5Hash);

after_login.http

# @name TENANTS
< ./hmack-auth.js
POST https://httpbin.org/post
Content-Type: application/json
Accept: application/json
Authorization: APIAuth {{SESSION_ID}}:{{SIGNATURE}}
X-Date: {{XDATE}}
Content-Md5: {{MD5HASH}}

{
  "data": {
    "tenants": {
      "id": "1234567890"
    }
  }
}

Warning

Make sure to run lua require("kulala").clear_cached_files() after the update
and also make sure you have installed crypto-js via npm install crypto-js --save-exact

I think you should also note that response.body.json.data.userSignIn.secret only works if the response body is json and contains a root element json if it just has data as root node, you need to access it via response.body.data.userSignIn.secret.

@hsanson
Copy link
Author

hsanson commented Oct 6, 2024

@gorillamoe thanks for the quick response and great support. I am almost there.

Now my server complains that the signature is incorrect that I believe is because the request body I get in the pre-script is not exact same as the body Kulala uses when sending to the server. I printed the body in my script and using :messages I can see it contains new lines and misses the GraphQL query and variables keys. This is what I add to the canonical string to compute the signature:

 query getTenant {^M                                                                                                                                                                                            
  currentUser {^M                                                                                                                                                                                                   
    id^M                                                                                                                                                                                                            
    tenant {^M                                                                                                                                                                                                      
      name^M                                                                                                                                                                                                        
      id^M                                                                                                                                                                                                          
    }^M                                                                                                                                                                                                             
  }^M                                                                                                                                                                                                               
} 

However, printing the curl command used for the request is clear that this differs from what Kulala is sending:

--data '{"query": "query getTenant { currentUser { id tenant { name id } } }", "variables": ""}'

Since the server computes the same signature using the received request body, if the body used by the client and the server is not exactly the same then the signatures will differ. The tryGetSubstituted() function should return exact, byte-by-byte, data that is being sent to the server in the request.

My full script for reference:

const MD5      = require('crypto-js/md5');
const Hex      = require('crypto-js/enc-hex');
const Base64   = require('crypto-js/enc-base64');
const hmacSHA1 = require('crypto-js/hmac-sha1');
const hmac256  = require('crypto-js/hmac-sha256');
const sha256   = require("crypto-js/sha256");

// Session ID and Token are set by `login.http`
const sess_id = client.global.get("SESSION_ID");
const sess_token = client.global.get("SESSION_TOKEN");
const url = new URL(request.url.tryGetSubstituted());
const method = request.method;
const path = url.pathname;
const typeHeader = request.headers.findByName('Content-Type')
const type = typeHeader.getRawValue()
const body = request.body.tryGetSubstituted();

const xDate = (new Date()).toUTCString();
const md5Hash = Base64.stringify(MD5(body));
const canonicalStr = [method, type, md5Hash, path, xDate].join(',');
const signature = Hex.stringify(hmacSHA1(canonicalStr, sess_token));

client.log("===============================REQUEST==================================")
client.log("Body " + body);
client.log("CanonicalStr " + canonicalStr);
client.log("Session " + sess_id);
client.log("Token " + sess_token);
client.log("Signature " + signature);
request.variables.set("SESSION_ID", sess_id);
request.variables.set("SIGNATURE", signature);
request.variables.set("XDate", xDate);
request.variables.set("MD5Hash", md5Hash);

@hsanson
Copy link
Author

hsanson commented Oct 6, 2024

Additional note: if I make a typo in the script name I get this error that is not very intuitive. Consider adding a check for the existence of the script and throwing a more intutive message before executing it.

Error executing vim.schedule lua callback: ...im/lua/kulala/parser/scripts/engines/javascript/init.lua:70: attempt to concatenate local 'userscript' (a nil value)                                                  
stack traceback:                                                                                                                                                                                                    
        ...im/lua/kulala/parser/scripts/engines/javascript/init.lua:70: in function 'generate_one'                                                                                                                  
        ...im/lua/kulala/parser/scripts/engines/javascript/init.lua:88: in function 'generate_all'                                                                                                                  
        ...im/lua/kulala/parser/scripts/engines/javascript/init.lua:122: in function 'run'                                                                                                                          
        ...l/share/nvim/lazy/kulala.nvim/lua/kulala/parser/init.lua:594: in function 'parse'                                                                                                                        
        ...ocal/share/nvim/lazy/kulala.nvim/lua/kulala/cmd/init.lua:87: in function 'run_parser'                                                                                                                    
        ...local/share/nvim/lazy/kulala.nvim/lua/kulala/ui/init.lua:204: in function <...local/share/nvim/lazy/kulala.nvim/lua/kulala/ui/init.lua:193>                

@gorillamoe
Copy link
Member

Now my server complains that the signature is incorrect that I believe is because the request body I get in the pre-script is not exact same as the body Kulala uses when sending to the server. I printed the body in my script and using :messages I can see it contains new lines and misses the GraphQL query and variables keys.

That is expected behaviour. I implemented it as per docs on intellij and they state it this way.

But I got curious on that and thought, maybe i misread that in their docs, so I went ahead and tried it in postman and even installed intellij webstorm to test that part.

They both handle it the way it's implemented in kulala.nvim at the moment.

image

If you need the "computed" graphql query in this case, it might be something completly different and would be just available in kulala and not work in other clients (which is not something I'm completely against).

I would propose something like this:

In case that it's a grapqhl request, the request.body would also have a getComputed() method, which would return the "raw" request body you see in the copy as curl command.

Would that suffice?

@hsanson
Copy link
Author

hsanson commented Oct 8, 2024

@gorillamoe thanks a lot!! works wonderfully. Here is my request and script in case anyone also needs to authenticate with servers using api-auth,

# @name TENANTS
< ./hmack-auth.js
POST https://{{HOST}}/graphql HTTP/1.1
content-Type: application/json
Accept: application/json
X-REQUEST-TYPE: GraphQL
Authorization: APIAuth-HMAC-SHA256 {{SESSION_ID}}:{{SIGNATURE}}
X-Date: {{XDate}}
X-Authorization-Content-SHA256: {{ContentHash}}

query getTenant {
  currentUser {
    id
    tenant {
      name
      id
    }
  }
}
const { createHmac, createHash } = require('node:crypto');

// Session ID and Token are set by `login.http`
const sess_id = client.global.get("SESSION_ID");
const sess_token = client.global.get("SESSION_TOKEN");
const url = new URL(request.url.tryGetSubstituted());
const method = request.method;
const path = url.pathname
const contentTypeHeader = request.headers.findByName('Content-Type');
var type = "application/json";
if(contentTypeHeader) {
  type = contentTypeHeader.getRawValue();
}
const body = request.body.getComputed();
const xDate = (new Date()).toUTCString();
const hash = createHash("sha256").update(body, "utf8").digest("base64");
const canonicalStr = [method, type, hash, path, xDate].join(',');
const signature = createHmac('sha256', sess_token).update(canonicalStr).digest("base64");

request.variables.set("SESSION_ID", sess_id);
request.variables.set("SIGNATURE", signature);
request.variables.set("XDate", xDate);
request.variables.set("ContentHash", hash);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants