Bug 64446 - cURL code generated for Post/Put requests from Postman is not imported correctly.
Summary: cURL code generated for Post/Put requests from Postman is not imported correc...
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 5.2.1
Hardware: PC All
: P2 normal (vote)
Target Milestone: JMETER_5.4
Assignee: JMeter issues mailing list
URL:
Keywords: FixedInTrunk
Depends on:
Blocks:
 
Reported: 2020-05-16 16:10 UTC by dineshbiradar95
Modified: 2020-08-01 20:56 UTC (History)
1 user (show)



Attachments
error_for_Delete_with_data-raw (175.90 KB, patch)
2020-06-11 14:23 UTC, dineshbiradar95
Details | Diff
query_params_added_in_path (218.28 KB, patch)
2020-06-11 14:23 UTC, dineshbiradar95
Details | Diff
Query parameter related attachment (218.28 KB, image/png)
2020-06-11 14:27 UTC, dineshbiradar95
Details
This attachment is related to error message during cURL import with Delete method. (175.90 KB, image/png)
2020-06-11 14:28 UTC, dineshbiradar95
Details

Note You need to log in before you can comment on or make changes to this bug.
Description dineshbiradar95 2020-05-16 16:10:20 UTC
Steps to Reproduce :
1. Refer the site - https://restful-booker.herokuapp.com/apidoc/index.html
2. Copy the "Auth - CreateToken" and import into Postman (refer link - https://stackoverflow.com/questions/27957943/simulate-a-specific-curl-in-postman)
3. Click on 'Code' option in postman and copy the cURL code for the above request. Sample as below - 
curl --location --request PUT 'https://restful-booker.herokuapp.com/booking/1' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Cookie: token=e316eaa04180abb' \
--header 'Content-Type: text/plain' \
--data-raw '{
    "firstname" : "James",
    "lastname" : "Brown",
    "totalprice" : 111,
    "depositpaid" : true,
    "bookingdates" : {
        "checkin" : "2018-01-01",
        "checkout" : "2019-01-01"
    },
    "additionalneeds" : "Breakfast"
}'
4. Open JMeter and go to Tools -> Import from cURL.
5. Enter above code snippet into the text area and click on 'Create Test Plan' button.
6. Observe that : Parameters and headers are not imported correctly.
Comment 1 dineshbiradar95 2020-05-16 16:17:59 UTC
Please refer this link for recording - https://drive.google.com/file/d/1axdjKC7TcgP6iG-BxQCb_-ej_xGbmipX/view?usp=sharing
Comment 2 Felix Schumacher 2020-06-07 09:18:24 UTC

*** This bug has been marked as a duplicate of bug 63452 ***
Comment 3 dineshbiradar95 2020-06-09 05:05:55 UTC
(In reply to Felix Schumacher from comment #2)
> 
> *** This bug has been marked as a duplicate of bug 63452 ***

Hey Felix,

- You marked this issue as Duplicate of Bug 63452 but 63452 is already marked as Resolved Fixed and it's fixed in JMeter version 5.2 and I am still not able to import cURL (with valid parameters) into JMeter.
- Based on the above could you please Reopen this issue or Bug 63452. Or let me know I will Reopen it.
Comment 4 Felix Schumacher 2020-06-09 09:55:04 UTC
Right, I initially thought the --data-raw option was the problem, but it seems, there is more to it.
Comment 5 Felix Schumacher 2020-06-09 10:01:29 UTC
The problem here is, that the parser doesn't remove the backslash at line ends correctly from the input for the CLI parser. It gets \--header and \--data-raw as options, which it won't accept.
Comment 6 Felix Schumacher 2020-06-09 10:29:37 UTC
Another problem is the option 'request' which is currently not supported (nor the short version X)
Comment 7 Felix Schumacher 2020-06-09 10:53:44 UTC
Both problems should be fixed now.

* Better support backslashes at line ends (when not in quote mode)
* support -X|--request together with --data (and their variants)

Can you test the next trunk or nightly build and report back?

(What is not supported yet is GET method together with --data argument)

commit 353ad79c5497c4d624f8b276580b6b50d45b27b9
AuthorDate: Tue Jun 9 12:49:00 2020 +0200

    Better parse curl commands with backslash at line endings and support PUT method with data arguments
    
    Bugzilla Id: 64446
---
 .../jmeter/protocol/http/curl/BasicCurlParser.java    |  6 ++++--
 .../org/apache/jmeter/curl/BasicCurlParserTest.java   | 19 +++++++++++++++++++
 xdocs/changes.xml                                     |  1 +
 3 files changed, 24 insertions(+), 2 deletions(-)
Comment 8 dineshbiradar95 2020-06-09 12:44:18 UTC
Hi Felix,

Thank you for accepting the issue.

I would love to test the basic test but I have not done the testing using nightly build so I am not sure how to do that.

I have checked the "apache-jmeter-5.3.1-SNAPSHOT.zip" for 2020-06-09, from this link - https://ci.apache.org/projects/jmeter/nightlies/ and found that issue was still reproducible (i.e. Post request data not imported correctly).

Could you please let me know how can I test this issue using nightly build?
Comment 9 Felix Schumacher 2020-06-09 14:04:32 UTC
The nightly will be generated in the night (well, the European night), so you would have to wait a few hours for the next one.

You can try a current build from trunk, which should be available under
https://builds.apache.org/job/JMeter-trunk/lastSuccessfulBuild/artifact/src/dist/build/distributions/

Or you could try building it yourself as described at https://jmeter.apache.org/building.html :)
Comment 10 dineshbiradar95 2020-06-10 05:23:21 UTC
Hi Felix,

I am referring the site https://restful-booker.herokuapp.com/apidoc/index.html for checking the basic scenarios for testing just in case if you want to try the scenarios.

I tested 2 scenarios Post, Put and observed that the all the Headers are not imported correctly.

Steps to Reproduce :
- Check the "Booking - UpdateBooking" scenarios from the above site. 
- It's Put request with Content-Type, Accept, Cookie headers. After importing the cURL data (both from the site and from Postman) the "Cookie" header is not imported for the request.
Comment 11 Felix Schumacher 2020-06-10 13:58:05 UTC
I take it, that you meant "not all headers are imported correctly" instead of "all headers are not imported (correctly)"?

To extract cookies from headers, you have to check the option "Add cookie header to Cookie Manager".
Comment 12 dineshbiradar95 2020-06-11 05:32:12 UTC
Hi Felix,

Yes, I mean "all headers are not imported" specially the Cookie one. But as you mentioned we need to check the "Add cookie header to Cookie Manager" in order to manage the cookies then it will work.

Tested : 
- Now, I am not able to see any issues while importing the Post and Put request through cURL code generated from Postman. I have also checked basic Get and Delete cURL import and it's also working.

Observations :
- As you mentioned "--data-raw" argument is not supported yet, when I try do that I see error message. I observed the same for Delete request(could you please confirm this)?


Questions/Issues :
- After importing request having query parameters all the query parameters are added into Path field in JMeter, they should be added in Parameters section. Is this expected behavior?
Comment 13 Felix Schumacher 2020-06-11 09:46:12 UTC
(In reply to dineshbiradar95 from comment #12)
> Hi Felix,
> 
> Yes, I mean "all headers are not imported" specially the Cookie one. But as
> you mentioned we need to check the "Add cookie header to Cookie Manager" in
> order to manage the cookies then it will work.
> 
> Tested : 
> - Now, I am not able to see any issues while importing the Post and Put
> request through cURL code generated from Postman. I have also checked basic
> Get and Delete cURL import and it's also working.
> 
> Observations :
> - As you mentioned "--data-raw" argument is not supported yet, when I try do
> that I see error message. I observed the same for Delete request(could you
> please confirm this)?

My remark was a bit misleading. First, I looked at old code (was in the wrong directory, which held an old copy). Second, this was true in combination with specifying a method other than POST.

If you see error messages, be sure to post them, as we can't see them otherwise. So we can't help.

> 
> 
> Questions/Issues :
> - After importing request having query parameters all the query parameters
> are added into Path field in JMeter, they should be added in Parameters
> section. Is this expected behavior?

How did you specify those parameters? (I guess, it is expected, but am not sure, what you have done)
Comment 14 dineshbiradar95 2020-06-11 14:22:03 UTC
Hi Felix,

- Please see the screenshot error_for_Delete_with_data-raw.png for error message. The cURL is generated from Postman (not sure why it gave "data-raw" param value as empty). But after entering some value for "data-raw" field there is no error. So, looks like issue related to bad cURL data.
- Please see the screenshot query_params_added_in_path.png for point regarding query parameters added in Path section instead of Parameters section. From users point of view it will be good if those get added in Parameters section so that if there are multiple query parameters and we want parameterize them then it will be easy to manage. 
- If you want to file separate improvement for above query param problem then please let me know I will file it.
Comment 15 dineshbiradar95 2020-06-11 14:23:12 UTC
Created attachment 37305 [details]
error_for_Delete_with_data-raw
Comment 16 dineshbiradar95 2020-06-11 14:23:54 UTC
Created attachment 37306 [details]
query_params_added_in_path
Comment 17 dineshbiradar95 2020-06-11 14:27:21 UTC
Created attachment 37307 [details]
Query parameter related attachment

Please ignore the above attachment.
Comment 18 dineshbiradar95 2020-06-11 14:28:47 UTC
Created attachment 37308 [details]
This attachment is related to error message during cURL import with Delete method.
Comment 19 Felix Schumacher 2020-06-14 13:10:32 UTC
Thanks for the detailed information.

The used parser inside JMeter seems to have a bug, that it will not accept empty arguments for long parameters (as --data-raw). This needs more work, than I anticipated, so it might take longer to fix.

The other problem you described (query parameters not getting parsed into separate fiels) might be better placed in a new bug (enhancement, really), as it probably works as designed.
Comment 20 dineshbiradar95 2020-06-15 04:57:57 UTC
Hi Felix,

- Thank you for the update.
- Jfyi, regarding query parameter parsing behavior I have filed Enhancement Bug 64523
Comment 21 Philippe Mouawad 2020-07-26 11:59:41 UTC
Hello Felix,
Could you create a ticket for what has been fixed and what still remains to be fixed ?

It's not clear for me.

Thanks
Comment 22 Felix Schumacher 2020-08-01 14:39:39 UTC
I have create a new bug Bugzille 64636 for the problem with the blank argument. The other open issue is Bugzilla 64523 where the query parameters should be added to the parameter table instead of the path string.

As all other problems reported in this issue are fixed in trunk (as I believe), I am closing this issue.