Great article! Thanks! Actually we just sent a paper trying to start a deeper line. You mentioned documentation QA which is a great application, however if we go now to a big project that changes rapidly due to agile practices many times you will find the case that the documentation is outdated with respect to the last version of the code. So we are trying to actually do QA using the code and static analysis to reduce the context we pass if the code is too big. Of course this alao has its limitations, for now we teste it in a small microservices project and we were surprised that from 33 designed complex questions it was able to anseer the majority of the questions correctly where the questions went from purpose of an endpoint to something like give me the chain of calls from endpoint A to endpoint B or tell me if you see cyclic dependency etc. Of course this we submit is baby phase, hopefully the monsters will be able to give it a great use.
You know I love the connection between formal systems and machine learning so this kind of approach of doing as much as possible with formal analysis prior to asking a language model is really exciting for me. Love to see what comes out of that research. Please share the paper whenever you can ;)
Great article! Thanks! Actually we just sent a paper trying to start a deeper line. You mentioned documentation QA which is a great application, however if we go now to a big project that changes rapidly due to agile practices many times you will find the case that the documentation is outdated with respect to the last version of the code. So we are trying to actually do QA using the code and static analysis to reduce the context we pass if the code is too big. Of course this alao has its limitations, for now we teste it in a small microservices project and we were surprised that from 33 designed complex questions it was able to anseer the majority of the questions correctly where the questions went from purpose of an endpoint to something like give me the chain of calls from endpoint A to endpoint B or tell me if you see cyclic dependency etc. Of course this we submit is baby phase, hopefully the monsters will be able to give it a great use.
You know I love the connection between formal systems and machine learning so this kind of approach of doing as much as possible with formal analysis prior to asking a language model is really exciting for me. Love to see what comes out of that research. Please share the paper whenever you can ;)