chainlink gearWorkshop with Patrick Collins - Chainlink Spring 2023 Hackathon

Ethiopian Case Study – GBADS – Global Burden of Animal ,Major outputs The Global B urden of A nimal D isease s GBADs Ethiopia case study in collaboration with the different themes of GBADs will produce the following major outputs in the current phase -: GBADs will be known, understood, supported, and utilized by stakeholders beneficiaries in Ethiopia.; Livestock production systems in Ethiopia will Ripple crushes SEC’s supplemental authority letter, XRP price 1 day ago · Ripple calls the Judge’s attention to Upton v. SEC, a case that the regulator excludes from its list of legal precedents. The defendants argue that this is the closest case based on both fact chainlink gear Workshop with Patrick Collins - Chainlink Spring 2023 Hackathon
chainlink gear O super iate de US$ 500 milh... Workshop with Patrick Collins - Chainlink Spring 2023 Hackathon
foreign hello hello welcome everybody we are here for the security and auditing portion of the chain link hackathon I hope everybody is having a phenomenal come on hope youre all doing well I know youre all doing well youre entertain the hackathon hope youre all meeting each other hope youre all making friends its not the solidity its the friends we made along the way thats the real treasure right gmgm let me get let me see some GMs in the chat what up GM GM how we doing well give everybody a couple minutes to come on in and while youre coming in if you want to post in the chat here what your level of coding prowess is what your how strong of a coder You Think You Are how you feel how you feeling are you like Oh Im a wizard if youre a wizard smart contract engineer put a wizard in there if youre intermediate put an intermediate and then if youre kind of a beginning be like ah just kind of get started if youre like how the hell did I end up on the stream you know post that in the chat as well let me this will give me a good idea as to how to tailor this stuff oh you like The Foundry video Tippy appreciate it Half Stack beginner Okay cool so yeah give me your solidity level in the chat here and then also your smart cross trick auditing or security engineering level this will also give me a good idea of where to start Mario hello Im starting the nft on chain in your two-day course excellent excellent all caps love to see that intermediate beginning to Mid okay expert to Doc skimmer okay who here would say that they are kind of beginner-ish on security auditing whats your level of security and auditing what is your level of security and auditing heads up there Security on the Zero okay expert doc skimmer excellent a couple people following the free code Camp course excellent youd love to see that youd love to see that anyone else these docs are a little bit or these comments are a little bit delayed here beginner out of the securian boot camp last year excellent beginner level one beginner Okay cool so this will be perfect so this is more for beginner and intermediate security people so you are all in the right spot or even beginner and intermediate facility peeps so let me change my camera here obs a little bit of magic wonderful so quick introduction so my name is Patrick Collins I should be on this side here somewhere if I scroll all the way to the bottom uh Im one of the co-founders of the siphon protocol we do smart contract audits security and really our entire aim is to just help web3 in any way shape or form that we can right help web3 scale by enabling developers such as yourself to to do more so and in this presentation today were going to be going over auditing smart contract security how to find bugs just kind of the real beginners of the basics of security in web3 and security and smart contracts so thats what were going to learn today and how to find every bug literally the exact steps that the top Auditors in our space use to find bugs and secure their code so lets jump into it and were going to be doing some exercises did I spell that right exercises I dont think I did but its fine some exercises and Foundry if you want to look up like GitHub Ive got this link um then Patrick obviously slash Denver security this is a presentation I did at uh for especially for some Denver people um so if you want to follow along this is the repo that were going to be working out of um and it has all of the code that were going to be working with and its a QR code if you want to scan the QR code to get the link lets get froggy yes you love to see it you love to see it um uh but yeah so this is what were going to be doing were gonna be working out of this code base here uh oh yeah a little bit more about me my name is Patrick Collins More Country engineer auditor educator et cetera uh you can find my Twitter YouTube um our siphon website there as well um a lot of people have been saying theyre doing some of my free code Camp stuff yeah I love making educational content love helping developers at scale so so heres our agenda here a quick primer on why security is even important and then how long do we have is this an hour long Yep this hour long okay cool quick primer on why security is so important how to have a security mindset some prerequisites to getting into security and then were going to go into the audit process and a lot of the tools the tools bit is going to be probably the most important piece of this entire presentation uh and definitely the takeaway that I want you to um to have for this when it comes to tools these are kind of the steps that I like to think about people should really take so they should absolutely start with your test Suite then move to static analysis then fuzz testing differential testing from a verification AI tools Etc theres some other tools along the way well actually I guess I should say theres we would stick manual review in here but if youre like what the heck are all these words thats fine well get to this in a minute so Ive already did a little bit of a quick roll call of everyones experience level with security but yeah Im monitoring the chat I can see everyones comments here uh yeah if you want to put in the chat again like your confidence level when it comes to Smart contracts and your confidence level when it comes to security that would be awesome uh just so thatll give me more context as to how to tailor the rest of this presentation but so lets jump into why security right why do we care like why does this even matter well uh matters for a lot of reasons number one these numbers are quite big these are very big numbers so this is a snapshot of the site rect dot news and they keep track of top hacks that have happened in the web3 space uh all of these are more than half a billion dollars which uh if you look over here you see this little unaudited keyword it means that these were smart contracts that were unaudited and they just shipped it and they were like yeah these are probably fine and they were not fine and they lost over half a billion dollars because of that in 2022 alone defy was hacked for three billion dollars three billion dollars thats a lot of money and if you think about this okay if if the risk versus reward is okay I can either get a 200 million dollar hack or I can spend two million dollars in security if youre a protocol obviously spending two million dollars in security gives you a 99 reduction in costs isnt that amazing so using making sure your code is secure is going to help you in the long run for a billion billion reasons and its going to help web3 and youre going to look smarter and youre going to have a better reputation theres a million reasons why we absolutely need to focus on security in web 3. security is for both protocol devs and for Auditors okay its for both the protocol whoevers building the protocol its for them and for the Auditors uh creating it so if you want to be a auditor you should learn security if you want to become a smart contract developer you should learn security right security is for everybody you the reason especially its for protocol devs is you have to have a security mindset from day one you cant just like build your protocol and then at the end go hey all right cool auditor now make this secure because if your architecture is garbage from day one well I got news for you I dont care how good the auditor is if your architecture is garbage youre not going to get off the ground and this is kind of a funny example Id like to give lets say that uh you know this is your contract this car in the left is your contract and you go hey were done with our code auditor can you please make sure this is ready to drive can you please make sure this is drive safe um you know the auditor is going to turn around and go oh uh no uh I dont even think this is possible right the auditor would literally have to build the car for you so um we need to have Security in our minds from day one okay cool uh and also theres some jobs and stuff um if for those who want who do want to become smart country Auditors you become an auditor auditor at a firm an independent auditor tooling developer um this right here is kind of some salaries for a very senior security engineer um kind of a more mid-level is going to be around the 150k um more junior is going to be 100K um but so if you get really good at Smart Security theres a lot of uh theres a lot of careers available in this role as well so not only do you keep your protocol more safe theres some careers that you can go into too so uh lets talk about some prerequisites before you can really even go deep into solidity or its gonna be deep into security and auditing you definitely need some evm knowledge uh like solidity Viper whatever you want to be working with but even more than that you need a willingness to be constantly learning hackers are going to continuously change the way that they approach hacking protocols so you always need to be learning and always need to be growing when youre a security engineer and when youre doing security so if you dont want to learn anything youre probably not going to go very far but if you like learning if you love constantly diving into projects you love constantly scaling yourself up this is really all you need to start working about start working in security start thinking about security uh but the more the better right so having an adversarial mindset some industry specific knowledge doing some ethical hacking use of a security tools these all can help you become a better smart contract auditor and especially those security tools is something were going to be talking about today so this was kind of just a primer on security you know what it takes to become an auditor or security engineer are there any questions so far and Im down to pause for a second answer any questions oh thats the wrong camera lets start my other camera my good camera back on huh foreign are there any questions so far yeah it looks like I messed up my camera here whoops ah there we go any other questions so far I will wait for a question after this beginner friendly course do we have the knowledge to start auditing you have the knowledge to start thinking about auditing thats what I would say I will give you some paths as to where to go um if you just watch this I would definitely not recommend you leave this presentation and start your own auditing firm I think that uh you wont quite be ready for there but you will definitely have the knowledge to start thinking about moving in that direction absolutely you will have a high level overview of everything you need to be thinking about how are you Im doing great thanks for asking all right cool lets flip back cool all right so lets talk about the audit process whats the auto process look like if you finish your code once you finish your smart contract code oh am I doing this right once you finish your smart contract code oh actually two more questions are there resources we can use Knowledge from this Workshop to go and check out previous audits to learn from yes absolutely I will give you a ton of resources at the end of this a couple of them actually I can give you right now you wanna see kind of some of the the top audit reports and compare them against each other you want to search through all auto reports um uh check out this is a great place wow they totally redid the UI this looks awesome um check out this is a great place to check out all reports in one place like it literally says right here also if you want to start testing yourself you go to ethernet its a really good question either not by open Zeppelin its a list of its a list of games that you can try out to test your security another one is damn vulnerable defy Temple D5 by tencho this is another really good one that you can actually start testing out your skills great question and then let me punch in my test password sorry one sec Im not going to use that but whatever um all right cool great questions how do you motivate yourself motivate yourself to audit projects all the time well I would like web3 to be safer and that motivates me a lot oh also you can do competitive Audits and we can talk more about that later how can I know my smart contract is ready to be audited great question we will that question will be answered at the end of this um good question all right cool so lets talk about the audit process so first off there is actually no Silver Bullet to Smart contract auditing right however these are kind of the main steps that people usually take its some type of manual review and then also using tools to augment their review manual review is exactly what it sounds like youre going through the code and even more importantly the documentation and understanding what the protocol should do a lot of people think you need to be like an evm Savant to do audits but really you just need you almost more importantly need to be a business logic savant most bugs that we actually find come from some business logic implementation being done wrong and the only way you could catch that as a bug is if you understand what the code should be doing you understand the business logic so a lot of tools actually have a really hard time finding most bugs and actually theyve done studies and they found that 80 percent of all bugs are actually whats called machine unauditable so automated tools wont catch 80 percent of the bugs out there so meaning automated tools can really only catch twenty percent of all bugs crazy statistic which means manual review is going to be the tool thats going to catch at least 80 now AI is getting better um but if you try if you if you try using AI as kind of your main automated tool its not really going to help that much weve been experimenting with AI most auditing firms are experimenting with AI right now they can be a little bit helpful with giving you context but they kind of suck by themselves right now so yeah 80 of all bugs are whats called machine unauditable which is kind of a crazy step uh which uh brings kind of to this um yeah you got to read the docs you gotta actually understand what the protocol does and I know as developers were like ah we dont really want to read the docs were going to browse stack Overflow or chat DBT or whatever for hours and do that instead when it probably would have been better if we just read the docs um yeah and just trying random stuff its uh you gotta understand the docs youve got to read the docs so I just want to reiterate you need to understand what the protocol should do right because 80 of the bugs are going to be from machine unauditable issues understanding what the protocol does is one of the most important parts of manual review so what does that involve it involves you sitting your butt down and you reading a ton of code and words its a long process its a hard process but that is the main piece of manual review and with manual review repetition is the mother scale the more you do it the better you will get thats really it it see it might seem daunting at first it might seem incredibly hard and it probably will be incredibly hard at first but the more you do it the better you get right like I can come to a code base now um seeing you know 4 000 lines of code and know that okay after a couple weeks Im going to understand how every inch of this thing works whereas in the beginning when I was first starting I would I would see four thousand lines of code and I would go oh my God Im never going to understand any of this so repetition is a mother scale the more you do it the better you will get now this brings us to our first actual code piece here so this is a smart contract called caught with test we have this function set number and what this is supposed to do its supposed to set this number variable to whatever new number is if you saw this code if you saw this contract and youre a smart contract daughter and youre doing security you might go oh yeah this theres nothing wrong with this function right I dont see any solidity issues I dont see anything wrong but if you knew that set number should set number to whatever new number is you would immediately go Oh oh my goodness youre setting number to new number plus one thats wrong thats potentially a security issue right because this set number function is not doing what this protocol is intended to do and this would come from manual review but this is why testing are is so important so if we go to GitHub repo here and here we have a test folder caught with test if we were at a Foundry test where we just simply have hey okay my number equals 55 lets call set number once we call set number my number should equal that number in the contract this would fail and we can actually run this test this if we run Forge test Dash m R run that specific test oops this should fail and we do indeed see it fails here right because we should be doing if we do that uh this should be not number equals the number of plus one that should be number equals new number this and now if we rerun the test we actually get this passing fantastic Ill put it back to failing all right cool so annual review incredibly important um tests are a great way to augment your manual review to tell you hey heres what this code this what this what this function should do and its that combined with the docs that can actually help you do that so any questions about this so far this is kind of the most basic piece here well number plus equal number work uh you mean number plus equal new number um so so this is a really good question actually because again we need to understand what set number does right if were adding this new number to number then great yeah that would work but we would know from reading the docs oh set number should set new number to number so we know that this is not right and we would know that number plus equals number would also not work right because that isnt setting number thats adding to number does the audit often result in improvements in the protocol docs too as a side effect great question um this is something that I think isnt done enough um it should is the answer it doesnt always do that I think more audit firms do need to focus on almost like documentation audits as well because a lot of times protocols dont document what their code should do and that a makes the audit a lot harder and B can find issues right there so Matt to answer your question it doesnt but it should and more more security firms should definitely uh pay the attention to that thats like the logic in your favorite number tutorial right yes its very similar to that any other questions oh whats up greetings from Turkey oh hey didnt even see you there cool all right great were gonna keep going so lets talk about some tooling I already talked about manual reviews so were gonna skip over that um and we have an example that this just brings us to uh Denver security SRC account with manual review heres another good example of uh what a manual review would catch so this code by itself is fine theres something wrong with this right but we have this documentation here it says adds to to a number to add and returns it and we can clearly see that number to add adds only one right so an automated tool wouldnt catch this right um this is manual review and AI actually probably would be able to catch this which is great but we need to do number to add plus two instead of number to add plus one so this is something manual review and testing would catch the chat is way too lagging yeah the chats kind of lagging as an audit firm do you run their test cases before creating uh your ones uh thats a big step um in the auto firm theres gonna be in the auditing step often is yeah hey like do your tests even make sense and if Im specifically looking for bugs and I see and I run their tests and I find oh my goodness there dont have any tests on this lump of code well guess where Im looking for bugs I mean that lump of code thats exactly where Im looking for bugs or thats one of the first places Im looking for bugs so cool so we talked about test Suites a little bit we talked about manual auditing already a little bit but yeah test Suites are really you know one of the first lines of defense for any and every protocol you absolutely need to be having a test Suite if you if you come to Cipher and if you come to us and you say hey can you please audit this code we dont have a test Suite Im going to tell you youre not ready for audit so somebody said hey like how do you know youre ready for audit this is absolutely step one if you do not have a test Suite you are not working with a mature code base and you should not deploy just full stop you should absolutely under no circumstances deploy code without having a test Suite so just just full stop I will literally turn you away I dont care how much money you pay me I will turn you away because I know for a fact youre not ready for audit and my job as a smart contract auditor is not just to do an audit report my job is to make sure that youre more secure my job is to make sure that when you launch your protocol we can do it with a lot of assurance that were not going to get hacked and if you dont have a test Suite I have zero confidence zero confidence so always need to write a test suite and try to get as much coverage try to cover as many lines of code in that test Suite as possible so I have an example here this just brings us back to our cot with test that we already kind of went over right like this function will get caught really easily in a in a test Suite even the manual review we get costs really easily in a test Suite so this is absolutely 100 the first line of defense writing good tests is one of the most important steps in making sure that your code is ready for audit is there a way to download Foundry for WSL yes um you should be able to do should just be able to do this right here if you cant you can build from Source but this should work for WSL this line here uh what test Suite do I recommend I think any of these are great Foundry hard hat and brownie I would say are the most popular right now uh truffle and ape Brooks are great too for those of you who have been around a little bit longer daptools works really if you pick anything from this list great remix you can actually write your tests in solidity or JavaScript as well so any of these are any of these are fantastic choices its really just what you like the best so Foundry is the fastest framework by far all tests are written in solidity hard hat is the most widely used everythings in JavaScript brownie is pythonic everythings a python you can also use boa too if youre working with Viper um if youre using cuff youd probably want to use Foundry yeah Apex is pythonic remixes solidity or JavaScript and then truffle is Javascript based thank you great questions all right cool moving on static analysis static analysis is the process of automatically checking the code for issues without actually executing anything hence the debugging is static right so this is where you basically run some tool this is uh artificial intelligence would also be considered static analysis where you basically take some tool and you say hey look at this code does this look okay right um yeah youre gonna have to build from source so that that definitely sucks uh sorry to hear that um believe in my readme I dont care were gonna run this we have in here um uh yeah sorry most of these tools just kind of like dumbly look for keywords in specific orders but theyre great tools to use because they can just kind of be fantastic sanity checks and so heres the example were going to be working with its going to be this um with so we have this contract here and this is the age old issue here this is vulnerable to something called a re-entrancy issue this will compile fine you could absolutely find this in manual review and again actually everything could be found in manual review but we want to use tools that make give us some assurance that were going to make sure to find those right manual review we kind of have to trust human beings dont mess up tools will not mess up but this has something called the re-entracy issue down here in this line right here what happened is when we call this and pull the money out of this um they could send this to a contract that will re-enter this recheck the balance see that they have a balance withdraw again with jargon with jog in with jog in just keep basically withdrawing and then only update their bounces to zero at the end this would be mitigated if this line was up here because the balance is set up to zero and then make the call and if they try to re-enter their balance would be zero um real classic attack here if youre unfamiliar with re-entracy attacks if you Google it youll get a thousand results but feel free to ask any questions too but we can find this by just running a static analysis tool so theres slither.excluded oops uh so theyre not exclude dependencies well see that it actually will catch this for us so we dont have to trust that a human being is gonna um and then you can ignore that for now thats some thats symbol execution stuff for later I might have to oops okay there we go so info detectors reentracy and kawasal their dot withdrawal right so it found for us these re-entracy issues even gives it to us in red uh and so that we can know hey all you have to do is run Slither on this code base and we would have found this re-entracy issue re-entry is the issue unfortunately to this day is still one of the most common bugs we see in the wild and yeah all they would need to do a lot of the time is run Slither and it would have got it and its really disappointing so uh static analysis great tool to use any questions on static analysis any questions on static analysis which made a protocol fell for re-entracy uh a lot of them go to um go to leaderboard uh uh a lot of these so I figure which ones off the top of my head but a lot of these or read the cryptopians yeah the Dow hack this is commonly included in CI yes great question Evan uh yes a lot of big protocols include Slither and in CI what more can we achieve with it uh well if we go to uh Slither GitHub over here you can scroll down to all the different detectors that they have excuse me uh its in the wiki I believe in the wiki uh detector documentation you can see all the different types of detectors that they have here theres a huge huge huge list of stuff they detect for so you can find a lot of stuff in here its so that your preferred static analysis tool uh yeah I definitely I definitely pretty much always use it big scary yes does chain link have Labs have any rest calls available Ive only seen API calls uh um Im not sure I follow chain link uh you can make calls uh if thats what youre looking for yeah like I guess I dont know what you mean Ive only seen API calls like a rest call is an API call do I need to be a smart contract engineer first to be a smart country auditor no a lot of these four country Auditors that I know are mediocre smart contract Developers it definitely helps though it definitely definitely helps to be a smart country engineer yeah the crypto the Dow hack yeah the most infamous attack of all time any other questions these are great questions by the way cool a little bit delay on the uh comments here but thats fine uh fuzz testing fast testing also known as fuzzing involves providing random data as inputs during tests and in my opinion this is the brand new floor for security Ive got a great video uh oh can we go to YouTube there it is right there got a fantastic video on fuzzing this one right here I highly recommend everybody watch it I do some silly stuff in it as well it shows me the silly stuff oh cool uh we do some silly stuff in it as well but it gives a really fantastic overview of fuzzing and how it can be super super effective to making sure your code is safe so but yeah at the end of the day its really just sending random data as inputs to our smart contracts and this is even better than testing because oftentimes we cant think of all the different scenarios that an attacker might do so we can string together a whole bunch of random scenarios and use that as a basis to basically kind of cover our bases hey lets just do this whole bunch of random scenarios and see if any it breaks anything so if we come to our and then actually let me comment this out so that doesnt keep doing that if we come to our code here weve got this function we know that this function should never return zero right because the doxation ever returns zero right writing a unit test for this would be really hard hey how do I make sure this function never returns zero okay well I could pass zero into my number okay and write a testification for that then I could write a test case for one that I could write a test case for two right test k for three four five six seven Etc how do we make sure this never returns zero well what we can do is we can just keep passing a ton of random numbers to this um and let a process go that just kind of keeps picking different numbers and see if we actually break this property or invariant so this is our invariant of the system that this should never return zero and thats what we test for in our fuzz test so if we have in here call with photos.t.sol weve got a fuzz test written in Foundry and we have this test fuzz function which itll keep populating this random number with random numbers and then well keep calling do more math with random numbers and then well just make sure hey make sure it doesnt return zero right because it not returning zero is our invariant so what we can do pull this up we do Forge test Dash M paste this in run this fuzz test and once it compiles or decides to not compile whatever it wants to do I guess come on all right here we go come on okay man so no we wouldnt need vrf for this because we dont need a verifiably random number for this we just need a bunch of random-ish numbers um this isnt part of a contract why is this not looking to compile for me here yes out here we go come on Democrats have spoken my computer struggles running OBS and other stuff at the same time and thats something Im doing right now so its probably not thrilled about this foreign stuck somewhere waiting for my ape and Foundry vids excellent excellent excellent excellent oh maybe it wants us back in maybe its mad at me for removing that oh thatd be an interesting bug that Id have to report wow oh that is interesting Im gonna have to report that as a bug looks like if I put the model Checker in but I dont add contracts it gets mad at me uh well okay whatever um cool forwards test Dash m s fuzz now run the fuzz test and boom we found that right away it was able to find if we pass one two six five we actually break our assertion so if we go back to Kyle with fuzz if I do one two six five we can see we have this conditional here if my number equals one two six five return my number module one two six five which is what um 0 plus 1 minus one times one which would be zero so this is actually going to return zero if we pass one two six five so it was able to find a case where we returned zero and say oh we found a breaking use Case by just passing random numbers to this so obviously this is incredibly powerful here um cool so thats fuzz testing any questions about fuss this thing have you seen AI being used to pick out and variants from protocol docs contracts code and then to generate tests about its something that Ive been experimenting with a lot Ive seen other people too um its mediocre at best at the moment definitely been having a hard time especially when you get more complicated contracts most AIS have a context limit so if I you know dump four thousand lines of code and say hey like this is this massive contract uh can you give me the invariance its just not going to be able to process its just too much for it to handle so weve been experimenting with different approaches to kind of piecemeal it together results have been kind of inconclusive its something that a lot of people are working on though for sure great question Matt I think its I think its something were going to keep experimenting and exploring with but there may be certain complex cases where the conditions would break when we passed test cases in a Serial manner will fuzz detect that but there may be certain complex cases where the condition would break when we passed test cases in a Serial manner will fuzz detect that Im not sure I follow the question but I think I know how to answer what youre getting at so uh picking random numbers uh uh obviously a un256 has a massive amount of numbers here right so if we wait long enough in our fuzz test we could essentially just iterate through every single random number possible right and hopefully just cover all use cases right we just we just put in every single possible unit 56. thats kind of unfeasible for a Time perspective wed have to wait I dont even know how long probably decades centuries for our first test to complete and you know we need to ship so what most buzzers do is theyre actually whats called smart fuzzers and how they pick the random numbers actually will determine how good your fuzzer is so what most fuzzers will do and in fact what The Foundry photo does is it sees that theres this conditional in here and it goes huh theres this conditional pointing out one uh 1265 lets try that as one of our original random numbers lets try two lets try one lets try one two three four five lets try all of these as random numbers and lets see if that works so how your fuzzer picks random numbers and actually picks semi-random numbers will deter how good it is so itll usually do this thing called smart buzzing where to look at your code and and try to figure out what it should pass just to get a good starting point and then if none of those work itll go back to just picking randomish numbers however there are cases where fuzzing actually wont find certain issues and this is if you have a crazy crazy specific use case the smart fuzzer cant do it and you dont have you know 12 centuries to wait for your fuzz testing run and well get to that in a minute but great questions so far all right cool moving on feel free to keep asking questions uh next is another type of fuzzing but a lot of people refer to them as invariance because thats what Foundry calls them uh this is something called stateful fuzzing so in our previous example here uh as fuzz here we pick a random number we call do math we check the assert we and then we then we start over we redeploy caught with fuzz we pick a random number we do it we check it we start over were picking we deploy a new contract pick a random number start this is called State less fuzzing because every single time we run this test we start from brand new we start from a brand new state we dont have any state this is called stateless fuzzing State full fuzzing or founder calls them invariant tests is where the system remembers the state of the last fuzz run and continues with a new fuzz test so this is State full fuzzing and this is incredibly powerful because of this example were about to see here so lets say we have this once again we know our invariant is going to be should never return zero from this function do more math again and in here we we can start to figure out okay well how can this return zero so first we do this U into 56 my number divided by 1 plus my value stored value equals response response how can I make this be how can I make this return zero how can I break this invariant well this stored value up here starts off as 100 um excuse me this uh this my value up here starts off as 1. and if we do you know one divided by 1 this could be one plus one is two so response is two return response what we could do is we call new value whats zero over one is um if we if we change my value to 0 0 over 1 is 0 plus 0 is 0 return 0. in order for us to break this invariant we would first need to call change value by passing zero and then call do more math again right so this actually involves two steps so just us calling do more math again with random numbers here this will never break our invariant that we should never return zero because we need to call change value so with stateful so in order to break this invariant we actually need to pass not only random data in here but also random function calls with random data sorry so if we go to our test case for this its a little bit more advanced uh theyre called invariant tests in Foundry which are just stateful fuzzing what we do is we set up our caught with stateful fuzz contract which has that bug in it and we tell Foundry okay that contract is going to be our Target contract I want you to string together a ton of random function calls with random inputs to their input parameters in all the coming fuzz tests so and youll see that if we just do a regular State less fuss test itll actually pass it wont find the issue but if we do an invariant test or a state full fuzz test itll do a whole bunch of random function calls itll say okay Im going to change the value to zero and then Im going to call do math do more math again so now do Forge test Dash m invariant test math doesnt return zero its actually going to give us a string a sequence of function calls to break our invariant and make it so that that returns zero so it says Okay first were going to call change value and were going to pass zero and then were going to call do more math again and were going to pass really whatever we want and so fuzz our stateful fuzz testing was able to find this simple fuzzing I would argue is probably one of the most important testing methodologies especially in D5 its very easily able to find a lot of issues and I think especially every default protocol needs to be using fuzz tests and staple fuzz tests because youre just going to be have so much more confidence that your code is programmatically checked for and when Im doing an audit Ive actually found a bunch of bugs by just going ah this is kind of confusing Im just going to write a fuzz test suite and we literally like find bugs in the first couple days because we understand the invariance of the system we write some fuss tests boom catch bugs cool any questions about this is there manual fuzzing like uh what do you mean uh yeah I mean like maybe I picked some random data sure uh yeah you can do that is there some rule to follow which functions should be tested with fuzz tests yeah and this is just goes back to Properties or invariants of the systems something like in D5 you know maybe you have a stable coin and uh you just want to think what are the properties or the invariance of the system any property or the invariant of the system you want a fuzz test so lets say you have a stable coin an easy invariant might be the total supply of the stable coin should always be greater than the total collateral debt positions of the stablecoin or maybe an easier one you know like a lottery for example hey um there should always be one winner there should never be two winners right thats a thats an easy invariant two fuzz test for a lottery so then you would just say okay the lottery contract is my Target contract there should always just be one winner or maybe another environment might be um the winner should always uh collect less money than the total sum of entry tickets right because maybe if you find a way to steal more money from the protocol than was given an entry takes you could actually steal too much money from the protocol right so you want to fuzz test any property or invariant of the system great question fantastic question why cant we all just get along make memes and play WoW um I mean we can do that too but uh Im trying to build the future of Finance so uh should your fuzz assumptions match your contract error statements and more if applicable should your fuzz assumptions match your contract error statements and more if applicable Im I dont totally follow your question um should your fuzz assumptions I I think I hear what youre like should you have errors maybe is the question right because because maybe youre I think I think I understand the question maybe because maybe your property or invariant isnt like a oh if you know if token too much revert right its really kind of like more of an architectural system oh um here uh uh what do you mean match your contract error statements I dont I dont totally I think I think maybe is the answer so thats kind of a little bit more advanced for what were gonna go over here yeah so uh when you want to make sure your fuzz tests input parameters are within a certain range youll use VM assume VM bound um Im not sure if I follow the question if you want to rephrase for me uh so stateful fuzz is more better than stateless or they have their own efficiency in different scenarios so they have their own uh trade-offs right State full fuzzing takes a lot longer to write tests for um yes at the end of the day state full fuzzing is going to be more inclusive um and go over more stuff but say less fuzzing is going to allow you to kind of more hyper specifically test a specific a single specific contract right and this again goes to okay the random number selection right if you know if you just want to make sure that the input parameter to the input parameters to a function are good you just do a stateless fuzzing right if you do state full fuzzing then you might not cover the entire Suite of inputs that you want to cover so it they have trade-offs is basically um they have trade-offs were going to learn more about testing spr interesting Workshop um uh my YouTube channel go to Patrick Alpha C on YouTube weve got a more in-depth video on fuzzing symbolic execution um and some other stuff like that great questions everybody really good questions all right cool uh and we have five minutes left so differential tests were gonna skip this this is um if you want to learn more you can check out The Foundry docs were going to skip it and then finally form a verification uh formal verification or FV its kind of a generic term for applying formal methods to verify correctness of Hardware applying FM means anything based on mathematical proofs in software often used to often use as a proof of correctness or proof of bug and thats kind of a lot of big words um and I kind of like just to break it down um and say okay well uh formal verification is just converting code to math thats it once you convert your code to math you can solve your math like youd solve any expression uh there are a lot of tools that do this like Matt manticores uh Z3 is a um a solver uh Satora but yeah its really just taking your study functions and converting it down to math now symbolic execution now a form of verification in Spock execute excuse me and symbolic execution is a way to do formal verification now the downside of symbolic execution is they take a long time to set up correctly and you need to be you need to have a lot of domain expertise to get this right and most of the time isnt worth it um we have a video on this on my YouTube as well on formal verification right here lets go check that out its a lot of fun um but also uh theres some great threads kind of com uh uh theres some great tips in the description for the blog we wrote with it as well kind of explaining some of the best tools to do with it uh I would argue most of you probably arent going to need to do symbolic execution because a sufficiently powerful fuzzer is really gonna be all you need um we have a demo if you guys want to play with the demo in the get a repo go for it were running a little short on time so Im going to skip it now um but finally AI tools I talked about this a little bit before heres a a wonderful example of where AI can really not be good uh yeah this is whats heavier 10k of iron or 10k of cotton theyre the same but chechi PT is like Irons heavier right like some stuff like this is kind of funny that it can get wrong uh and thats it q a for the like two minutes that we have left let me switch my camera q a questions what specific strategies or algorithms are employed by Foundry to ensure that the generated test cases during invariant fuzzing provide extensive coverage of the softwares code base um so they again the The Foundry fuzzer is whats called a smart fuzzer right so they pick uh uh inputs based off of kind of what they read from the contract um what youre kind of asking is hey how do you ensure that the randomness use cases have good coverage did they cover enough use cases so I actually dont think Foundry has a really solid way of doing that right now um the consensus diligence buzzer does if you guys are looking for a paid um alternative so the consensus diligence filter will tell you hey weve covered 60 of all possible cases right which is really uh powerful right if youre looking for more assurance that your fuzzing is working um that might be a solution that youre looking for great question PC is at high man cool any final questions before we jump off here whats the difference in testing in Foundry versus hard hat are there any tests that work better on any of those so Foundry is definitely faster so if youre looking for Speed Foundry is definitely quicker um hard hat uh is written in JavaScript so if you like kind of JavaScript testing you can do that um its really up to you its what you like better but I mean thats kind of the main differences can you please provide more resources like damn vulnerable defy of auditing a sample contract Dan vulnerable D5 is good ethernet is good um theres some other capture the flags that are really good um Im trying to think of more um I cant think of any more off the top of my head but those are really good ones how to be better at auditing uh sit your butt down and audit a lot so competitive oh oh actually oh oh this answers both your questions uh competitive audits are great places to go to as well something like code Farina Sherlock the saloon code Hawks um those are all fantastic places those will allow you to do real Audits and if you do well you actually will get paid so the best way to become better auditor is to do a lot of audits so yeah reading reports is really good like solid it fantastic site you know so maybe you even crack open a code base and you do an audit yourself and then you compare it to an existing audit report right theres so many so much theres so much free resources out there for you to get better at auditing absolutely on the site from website or excuse me on the site from GitHub we have a a list of public auto reports that weve done if you want to compare if you want to grab the one of the hashes and see if you can find more bugs so great questions all right guys well uh we are at time here thank you all so much for coming I hope you enjoyed this presentation I hope you learned a lot about how to be a little bit more secure and I guess Ill say this passing or this final thought here retail is never going to come to web3 until we make web 3 more secure a six percent is a ridiculous statistic its like if you went to a bank in your country and said hey Id like to put my money in this Bank the bank went okay cool but theres a six percent chance that in a year it is all gone you would never put your money in that bank and its the same thing with D5 retail will never put their money in D5 if we keep getting hacked as we are so we need to make it more secure and we need to improve the narrative from hey just doing on it just getting on it to make web3 more secure and having that mentality on lets scale web3 for security is going to make us a lot better in the long run so great questions thank you all so much for coming and good luck on the rest of the hackathon bye Register for the hackathon to compete for $450K in prizes: Chainlink is the industry-standard Web3 services platform that has enabled trillions of dollars in transaction volume across DeFi, insurance, gaming, NFTs, and other major industries. As the leading decentralized oracle network, Chainlink enables developers to build feature-rich Web3 applications with seamless access to real-world data and off-chain computation across any blockchain and provides global enterprises with a universal gateway to all blockchains. Learn more about Chainlink: Website: Docs: Twitter: Chainlink Chainlink,