-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add installation truncation #192
Closed
Closed
Changes from 128 commits
Commits
Show all changes
134 commits
Select commit
Hold shift + click to select a range
977309e
add logic to write status complete file
feng-j678 012fb39
add logic for latest status complete file
feng-j678 86adb29
add truncation logic
feng-j678 4c182ea
change logic to binary search and trying utest
feng-j678 089fac6
add utest for truncated file, change status_complete*** to complete_s…
feng-j678 6cf7d99
add unit test to CoreMain for truncation and modify the load_status_f…
feng-j678 89f2d92
add unit test for testing errors for truncation
feng-j678 6cdce2b
Merge branch 'master' of https://github.com/Azure/LinuxPatchExtension…
feng-j678 83f7d17
remove json.dumps on message to avoid adding \ and remove logic for c…
feng-j678 ece5ad6
add logic for quote count after truncation
feng-j678 f213d6f
modify the quote_count logic and set_package_assessment_status to use…
feng-j678 7114d6d
fix file formating and add more tests for report_sequence_number_chan…
feng-j678 8242685
add more unit test
feng-j678 faa0bbc
remove unused variables in test_status
feng-j678 e9800ab
add truncation logic for installation
feng-j678 fdf7d8f
modify the set_package_install_status and classification to use map f…
feng-j678 1555c07
add unit tests for installation truncation
feng-j678 45823ac
add more unit test for testing installation truncation
feng-j678 3e1cd68
reset installation_tmp_map
feng-j678 6d6f8fa
add classification to test_installatio_keep_5
feng-j678 89e99da
revert set_pacakge_install_status to double forloops because values i…
feng-j678 beb92bb
change hashmap to orderedmap because existing utest check for order o…
feng-j678 9ea154e
add unit test for get latest complete status file
feng-j678 e0d19b6
add test for get latest complete status file
feng-j678 790bc13
add descriptions to truncation tests
feng-j678 cfec898
add unit test for ordered map
feng-j678 274d69b
add test for checking if complete status is dir
feng-j678 7fd4383
add unit test for orderdict()
feng-j678 eb353e6
Modify Agent status Constants and revert white spaces
feng-j678 0577a4d
this has assessment truncatin logic bad format
feng-j678 a5c82fa
revert status handler format to as master version
feng-j678 fcc738e
revert code format from master version
feng-j678 662b176
modify variable naming convention add unit for latest complete status
feng-j678 65ee40d
change map to use orderdict to preserve element ordering because of py27
feng-j678 65cf245
add unit test for ordered map
feng-j678 3d186b9
chnage test_value to patch_count_for_test
feng-j678 4b60b38
modify the tombstone according to classifications
feng-j678 3d4b97f
modify unit tests for new tombstone records
feng-j678 4782bc3
add unit test for complete status file is a dir
feng-j678 0178e9d
add unit test for log_truncated_removed_packages_if_any
feng-j678 668aed6
merge new changes from assessment_truncation branch
feng-j678 3a8d421
merge in changes from assessment_truncation branch
feng-j678 4f4456e
resolve unit tests failures
feng-j678 3a27c9e
resolved unit tests problems
feng-j678 aed5b9a
failing unit tests
feng-j678 5695532
debug why unit tests are failing in coremain
feng-j678 5683aa5
add runtime.stop() to fix failing
feng-j678 d5ee671
fix unit test by correcting message patch tombstone record
feng-j678 d41b75b
modify unit test to whether file size is > or < agent file size limit
feng-j678 64aafd1
to reuse __set_errors_json for truncation
feng-j678 25ee9b1
remove code parameter in recompose logic
feng-j678 8b5c729
remove checks for agent file size limit in unit test per summary patches
feng-j678 8ba655b
add get_index method for low pri
feng-j678 3b27791
add logic for high and low pri intallation
feng-j678 7b8246b
add method description to __get_installation_packages_index
feng-j678 510a6da
extract assessment tombstone into a separate method
feng-j678 02fd327
refactor truncation main logic
feng-j678 8bbb522
merge changes from master to fix ci.yml and update statushandler
feng-j678 67576b3
add code flow to write_status_file()
feng-j678 dfcc478
refactor assessment cod, remove quote_count and use fix escape byte
feng-j678 88b6137
Merge branch 'master' of https://github.com/Azure/LinuxPatchExtension…
feng-j678 57e1cf5
refactor __recompose_truncated_status_file
feng-j678 141e6c1
fix code to consider no assessment truncation, everything remains same
feng-j678 33fd85a
update truncation exception message unit test
feng-j678 2c64887
remove unncessary removed_packages_details method
feng-j678 716d1d3
refactor truncation main logic
feng-j678 9f600ed
add get status file w/o list method
feng-j678 e4f682b
rename truncation varibles
feng-j678 4a559bd
use while loop to checking size limit
feng-j678 3023038
test fail pipeline
feng-j678 0c0d8da
add low_index_pri logic, status unit tests passed
feng-j678 8f45390
add low_pri index logic, pass all tests
feng-j678 f4a99c0
update unit tests and code flow
feng-j678 0e8650b
add extra to bottom of status handler.py
feng-j678 a098abc
change orderdict to collections.ordereddict
feng-j678 e474c34
change get latest complete status file to remove older complete status
feng-j678 89b88ef
change forward slash to back slash in glob.glob:
feng-j678 6301415
remove unused imports
feng-j678 6342db0
merge in changes from complete_status_file_logic branch
feng-j678 9004e51
pr review with todos and comments
feng-j678 1aa276b
commit 1 -refactor to resuse substatus_message method
feng-j678 e65e8b4
commit 2 - rename variables
feng-j678 21f6394
commit 3 - refactor global varibles assessment, installation packages…
feng-j678 9e428d6
Merge branch 'master' of https://github.com/Azure/LinuxPatchExtension…
feng-j678 e798676
change size_limit to capacity
feng-j678 997c758
commit 2 combine assessment and installation logic into one
feng-j678 7b539a1
change keep latest complete status logic to keep 10
feng-j678 97727f1
remove the break logic from current_list - diff
feng-j678 f363707
chnage test_assessment_operation_success_truncation_over_capacity bec…
feng-j678 f90b2f8
fix __removed_older_complete_status_files bug
feng-j678 2c99f29
commit 1 - refactor var, method, and combin logic in __apply_truncati…
feng-j678 165ef8e
commit 2 add fake version to tombstone, modify __removed_older_comple…
feng-j678 3d7ca29
commit 3 update pesudo code
feng-j678 e6ae224
commit 4 remove assertion on complete status deletion becuase github …
feng-j678 6a8b0f1
refactor truncation logic to write_complete_status_file()
feng-j678 7cf753f
commit 2 refactor substatus_msg
feng-j678 75afd05
commit 3 add write status comment
feng-j678 e8305a5
commi t 4 refactor assessment installation variables names
feng-j678 a89510d
Merge branch 'master' of https://github.com/Azure/LinuxPatchExtension…
feng-j678 df93219
commit 1 move the max_list_capacity - keepfive logic to __apply_trunc…
feng-j678 0623c2f
commit 2 add utf-8 encode to unit test
feng-j678 6e55396
commit 3 change variables in __get_new_size_in_bytes_after_truncation
feng-j678 f8d5e46
commit 4 simplify recompose assessment and installation substatus msg…
feng-j678 cd87ebc
add extra space to eof
feng-j678 30c05d3
commit 1 change the while loop condition to status_file_byte_size > …
feng-j678 9245c3b
commit 2 update pesuedo code
feng-j678 22d6d65
commit 3 refactor methods decriptions
feng-j678 ba077d2
test self.assertEqual(len(message_patches), 6) is 6 not 7
feng-j678 5fc2dd9
change self.assertTrue(len(message_patches) >= 6)
feng-j678 28d81d7
add performance for truncation
feng-j678 a328491
update test_installation_truncation_with_only_install_packages_over_s…
feng-j678 1b52a78
correct end.time - start.time in unit test
feng-j678 71fdd6e
remove comments and refactor __update_substatus_msg_packages_and_errors
feng-j678 ac0db9b
commit 1 refactor variables, methods
feng-j678 200e37d
commit 2 add named parameters
feng-j678 40eac52
refactor code
feng-j678 cd30132
refactor __recompose_truncated_status_file
feng-j678 9d083f8
merge in changes from master
feng-j678 f34c848
reset code changes because branch is ahead of master, refactor variab…
feng-j678 63351e2
add unit test for validating the log message
feng-j678 d775cea
change write log to tmp and read from it
feng-j678 c2fdc5b
move write to temp log file into a method for unit test
feng-j678 442593c
refact substatus_msg_at_index to substatus_message
feng-j678 aa7fb97
add FORCE_WRITE_TRUNCATION constant, add check for 1 min timestamp co…
feng-j678 7120636
add unit test for 1 min delay
feng-j678 764b59b
add checker for delaying truncation logic and unit test
feng-j678 cf4b555
Merge branch 'master' of https://github.com/Azure/LinuxPatchExtension…
feng-j678 51a3d53
add new comments and reset truncation boolean in load_status_file_com…
feng-j678 6a92ec0
refactor method names
feng-j678 831cb70
initialize truncation_timestamp inside __check_truncation_time_passed…
feng-j678 2521e8a
move self.__is_file_truncated logic to __check_status_file_size_perfo…
feng-j678 d322898
extract tombstone logic into a variable
feng-j678 2d411e1
rename var and method, and set Constants.StatusTruncationConfig.NO_TR…
feng-j678 424953d
change NO_TRUNCATION_IN_X_SEC = -1 to ensure truncation happens
feng-j678 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
q: Wouldn't truncated packages be logged at the end of each status file write?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
koshy wanted them at the very end of operation to reduce the space on log file. I am thinking on removing this method because we are retaining up to 10 complete status.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@feng-j678 pending
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My opinion, this should be added for each status file write, post truncation, because that gives the details of what happened in each truncation (and thus status file writes). Will help in debugging issues in future. @kjohn-msft for more thoughts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: to be raised and discussed between @kjohn-msft, @feng-j678 and me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pending for koshy's input on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment is still open and needs feedback. @feng-j678 please make sure all existing comments are addressed