Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Fortify is a SCA used to find the security vulnerabilities in software code. I was just curious about how this software works internally. I know that you need to configure a set of rules against which the code will be run. But how exactly it is able to find the vulnerabilities in code.
Does anyone have any thoughts about this?
Thanks in advance.
HP Fortify SCA has 6 analyzers: data flow, control flow, semantic, structural, configuration, and buffer. Each analyzer finds different types of vulnerabilities.
Data Flow
This analyzer detects potential vulnerabilities that involve tainted data (user-controlled input) put to potentially dangerous use. The data flow analyzer uses global, inter-procedural taint propagation analysis to detect the flow of data between a source (site of user input) and a sink (dangerous function call or operation). For example, the data flow analyzer detects whether a user-controlled input string of unbounded length is being copied into a statically sized buffer, and detects whether a user controlled string is being used to construct SQL query text.
Control Flow
This analyzer detects potentially dangerous sequences of operations. By analyzing control flow paths in a program, the control flow analyzer determines whether a set of operations are executed in a certain order. For example, the control flow analyzer detects time of check/time of use issues and uninitialized variables, and checks whether utilities, such as XML readers, are configured properly before being used.
Structural
This detects potentially dangerous flaws in the structure or definition of the program. For example, the structural analyzer detects assignment to member variables in Java servlets, identifies the use of loggers that are not declared static final, and flags instances of dead code that will never be executed because of a predicate that is always false.
Semantic
This analyzer detects potentially dangerous uses of functions and APIs at the intra-procedural level. Basically a smart GREP.
Configuration
This analyzer searches for mistakes, weaknesses, and policy violations in an application's deployment configuration files.
Buffer
This analyzer detects buffer overflow vulnerabilities that involve writing or reading more data than a buffer can hold.
#LaJmOn has a very good answer, but at a completely different level of abstraction I can answer the question in another way:
Your source code is translated into an intermediate model, which is optimized for analysis by SCA.
Some types of code require multiple stages of translation. For instance, a C# file needs first to be compiled into a debug .DLL or .EXE, and then that .NET binary file disassembled into Microsoft Intermediate Language (MSIL) by the .NET SDK utility ildasm.exe. Whereas other files like a Java file or ASP file are translated in one pass by the appropriate Fortify SCA translator for that language.
SCA loads the model into memory and loads the analyzers. Each analyzer loads rules and applies those roles to functions in your program model, in a coordinated manner.
The matches are written into an FPR file, with the vulnerability match information, security advice, source code, source cross-reference and code navigation information, user filtering specification, any custom rules, and digital signatures all zipped into the package.
Also an addendum to #Doug Held comment above ... Starting with Fortify 16.20, SCA now supports scanning .Net C#/ASP/VB source code directly - no longer requiring pre-compilation.
Yes - Fortify SCA supports scanning Objective-C and Swift for iOS and about 20 other languages and numerous frameworks. See more in the Fortify SCA Data Sheet:
https://www.hpe.com/h20195/V2/GetPDF.aspx/4AA5-6055ENW.pdf
You can also leverage Fortify SCA through via SaaS at Fortify on Demand and have experts run the scans and audit the results for you:
http://www8.hp.com/us/en/software-solutions/application-security-testing/index.html
Related
I’m trying to use Lua/Moonsharp API scripting. There’s a command library, but there isn’t a function that I need. My question is, am I able to create my own function, or am I limited to what’s been written?
Specifically, for a software called BobCAD a Lua plugin is available. There are lists of commands like Bcc.SetCamObjParameter, though, there isn’t the command that I need. I’m assuming that some aspects of the BobCAD software are inaccessible to API, but am I limited to the library in this plugin, or can I add my own commands? I thought that there would be some C# file somewhere in the program directory where I can read the functions and possibly learn how to create my own, but I don’t see anything like that. (Or are scripting functions set up internally to the software, and I’m only given access to what has been provided?)
(N.b. I'm not familiar with BobCAD specifically, but this answer should be generally applicable.)
Generally speaking, for use cases like this (where Lua scripting is provided as part of a larger program, or by a plugin for a larger program), it's the developer of the program or plugin who decides what API is available to Lua. In the latter case they may also be limited by what the program's plugin API allows; in the case of BobCAD you are likely limited to what the BobCAD plugin API permits, and then further limited by what parts of that API the developers of the Lua plugin chose to make available to Lua itself.
You can of course write your own functions in Lua, but in terms of actually talking to the host program you are restricted to the API that program makes available, unless they make available some mechanism (like LuaJIT's Foreign Function Interface) for reaching into the program from Lua and calling functions that were not made explicitly available -- which most do not.
As for finding "a C# file somewhere in the program directory" -- C# is a compiled language; C# libraries are generally shipped as pre-compiled CLR binaries (with a .dll extension) and do not contain source code. If the source code for BobCAD and/or the Lua plugin is available you could always modify it and re-compile it, but as BobCAD is commercial, closed-source software, I would not expect it to be available.
At that point, your options are basically:
figure out a way to do what you need with the commands that are available;
try to reverse engineer enough of the program to do what you need without access to the source code;
or look for another program that has the features you need.
I came across with a new dynamic language. I would like to create a coverage tool for that language. I started reading the source code of Perl 5 and Python coverage modules but it got complicated. It's a dynamic scripting language so I guess that source code of static languages (like Java & C++) won't help me here. Also, as I understand, each language was built in a different way and the same ideas won't work. But, the big concepts could be similar.
My question is as follows: how do I "attack" this task? What is the proper workflow I need to follow? What I need to investigate? Are there any books or blogs I can read about those kind of stuff?
There are two kinds of coverage collection mechanisms:
1) Real-time sampling of the program counter, typically by a clock running at 1-10ms. Difficulties: a) mapping an actual PC value back to a source line, b) sampling means you might not see execution of a rarely used bit of code, so your coverage reporting is inaccurate. Because of these issues, this approach isn't used very often.
2) Instrumenting the program so that it collects coverage as it runs. This is hard to do with object code... a) you have to decode the instructions to see where to put probes, and this can be very hard to do right, b) you have patch the source code to include the probes (this can be really awkward; a "probe" might consist of a 5 byte subroutine call but the probe has replace a single-byte instruction). c) you still have to figure out how to map a probe location back to a source code line. A more effective way is to instrument the source code, which requires pretty sophisticated machinery to read source, make probe patches, and regenerate the instrumented code for execution/compilation.
My technical paper Branch Coverage for Arbitrary Languages Made Easy provides explicit detail for how to do this in a general way. My company has built commercial test coverage tools for a wide variety of languages (C, Python, PHP, COBOL, Java, C++, C#, ProC,....) using this approach. This covers most static and dynamic languages. Some dynamic mechanisms are extremely difficult to instrument, e.g., eval() but that is true of every approach.
In addition to Ira's answer, there is a third coverage collection mechanism: the language implementation provides a callback that can inform you about program events. For example, Python has sys.settrace: you provide it a function, and Python calls your function for every function called or returned, and every line executed.
The fslex and fsyacc tools currently require 2-stage compilation, generating files that are then compiled by fsc. It seems to me that these tools would be much easier to use if the source files were embedded resources, fed to fslex and fsyacc programmatically and the generated code compiled on-the-fly using the CodeDom.
Is this feasible and, if so, what would be required to implement this?
Jon, this is a great question; in fact, one of the design goals I have for fsharp-tools (new lexer- and parser-generator implementations for F#) is for them to be embeddable, specifically to enable scenarios like this.
As of now, I haven't implemented (yet) the functionality which would let you do this easily in fsharplex, but don't let that deter you; I've written fsharplex (and the other tools in fsharp-tools) in a more-or-less purely-functional style, so there shouldn't be any issues with global state or anything like that. It should be relatively straightforward to hack up the compiler code so you can build a regex AST using some combinators, run the compiler to get a compiled DFA, then emit IL for your state machine into a dynamic assembly (which you could then "bake" and execute).
fsharpyacc currently uses an approach where I've put the bulk of the compilation logic into a purely-functional library, Graham; the idea there is that the grammar analysis/manipulation and parser DFA compilation algorithms should be generic, reusable, and easy to test, so anyone else wanting to build language tools with F# will have a common framework on which to build them. Likewise, contributions/improvements to Graham can easily flow back to fsharpyacc. Eventually, I will modify fsharplex to use this same approach, which will allow you to embed the regex compiler in your own code simply by referencing the NuGet package (you'd just need to write the code to generate IL from the DFA).
fsharplex and fsharpyacc use MEF to allow various backends to be plugged in; for now, they're only targetting fslex and fsyacc for compatibility reasons, but I'd like to implement code-based backends (as opposed to the current table-based backends) to get better performance in the future.
Update -- I just re-read your question and noticed you want to embed the *.fsl and *.fsy files themselves and invoke the respective compilers at run-time. You could accomplish this by compiling the tools and referencing the assemblies from your own projects. IIRC, I exposed an entry point in both compilers so they could be called from outside code; the main entry points (e.g., what gets executed when you invoke the tools from a console) simply parse the command-line arguments then pass them into this "external" entry point.
There is one problem with directly embedding the *.fsl and *.fsy files though; if you embed them, then run them through fsharplex and fsharpyacc at run-time, your user-defined actions (e.g., the code executed when a lexer or parser rule is matched) will still be specified as F# source code -- you'd need to decide how you want to compile them into executable code.
It should be feasible to provide a parser combinator-like interface with a backend that uses expression trees (the LISP "eval" of F#) or something similar, for full integration with the language. Or else a TypeProvider. There are many options. If table generation is an expensive computation, it could be cached by providing a Cache, for example a disk cache.
I think nothing except lack of time, dedication and expertise, prevents us from having tools with (non-monadic) parser combinator-like interface, yet efficient compiled implementation.
Sometimes I get back to this pet project of mine, playing with an algebraic approach to optimizing regular expressions (and lexers) specified in source using combinators and then compiled to a state machine. It still lacks a few key pieces for efficiency, but there it is:
https://github.com/toyvo/ocaml-regex-algebraic
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I want some examples. I always get confused, so with some examples I might be able to figure it out better.
Also: Is Eclipse an API or IDE?
An IDE is an integrated development environment - a suped-up text editor with additional support for developing (such as forms designers, resource editors, etc), compiling and debugging applications. e.g Eclipse, Visual Studio.
A Library is a chunk of code that you can call from your own code, to help you do things more quickly/easily. For example, a Bitmap Processing library will provide facilities for loading and manipulating bitmap images, saving you having to write all that code for yourself. Typically a library will only offer one area of functionality (processing images or operating on zip files)
An API (application programming interface) is a term meaning the functions/methods in a library that you can call to ask it to do things for you - the interface to the library.
An SDK (software development kit) is a library or group of libraries (often with extra tool applications, data files and sample code) that aid you in developing code that uses a particular system (e.g. extension code for using features of an operating system (Windows SDK), drawing 3D graphics via a particular system (DirectX SDK), writing add-ins to extend other applications (Office SDK), or writing code to make a device like an Arduino or a mobile phone do what you want). An SDK will still usually have a single focus.
A toolkit is like an SDK - it's a group of tools (and often code libraries) that you can use to make it easier to access a device or system... Though perhaps with more focus on providing tools and applications than on just code libraries.
A framework is a big library or group of libraries that provides many services (rather than perhaps only one focussed ability as most libraries/SDKs do). For example, .NET provides an application framework - it makes it easier to use most (if not all) of the disparate services you need (e.g. Windows, graphics, printing, communications, etc) to write a vast range of applications - so one "library" provides support for pretty much everything you need to do. Often a framework supplies a complete base on which you build your own code, rather than you building an application that consumes library code to do parts of its work.
There are of course many examples in the wild that won't exactly match these descriptions though.
The Car Analogy
IDE: The MS Office of Programming. It's where you type your code, plus some added features to make you a happier programmer. (e.g. Eclipse, Netbeans). Car body: It's what you really touch, see and work on.
Library: A library is a collection of functions, often grouped into multiple program files, but packaged into a single archive file. This contains programs created by other folks, so that you don't have to reinvent the wheel. (e.g. junit.jar, log4j.jar). A library generally has a key role, but does all of its work behind the scenes, it doesn't have a GUI. Car's engine.
API: The library publisher's documentation. This is how you should use my library. (e.g. log4j API, junit API). Car's user manual - yes, cars do come with one too!
Kits
What is a kit? It's a collection of many related items that work together to provide a specific service. When someone says medicine kit, you get everything you need for an emergency: plasters, aspirin, gauze and antiseptic, etc.
SDK: McDonald's Happy Meal. You have everything you need (and don't need) boxed neatly: main course, drink, dessert and a bonus toy. An SDK is a bunch of different software components assembled into a package, such that they're "ready-for-action" right out of the box. It often includes multiple libraries and can, but may not necessarily include plugins, API documentation, even an IDE itself. (e.g. iOS Development Kit).
Toolkit: GUI. GUI. GUI. When you hear 'toolkit' in a programming context, it will often refer to a set of libraries intended for GUI development. Since toolkits are UI-centric, they often come with plugins (or standalone IDE's) that provide screen-painting utilities. (e.g. GWT)
Framework: While not the prevalent notion, a framework can be viewed as a kit. It also has a library (or a collection of libraries that work together) that provides a specific coding structure & pattern (thus the word, framework). (e.g. Spring Framework)
Consider Android Development:
IDE: Eclipse etc..
Library: android.app.Activity library (Class with all code)
API: Interface basically all functions with which we call
SDK: The Android SDK provides you the API libraries and developer tools necessary to build, test, and debug apps for Android
(----tools - DDMS,Emulator ----platforms - Android OS versions, ----platform-tools - ADB, ----API docs)
ToolKit: Could be ADT Bundle
Framework: Big library but more of architecture-oriented
In other words...
IDE
Even notepad is an IDE (not notepad.exe) - Software you write/compile code with is an IDE.
Library
A bunch of code which simplifies functions/methods for quick use.
API
A programming interface for functions/configuration which you work with, its usage is often documented.
SDK
Extras and/or for development/testing purposes.
ToolKit
Tiny apps for quick use, often GUIs.
GUI
Apps with a graphical interface, requires no knowledge of programming unlike APIs.
Framework
Bunch of APIs/huge Library/Snippets wrapped in a namespace/or encapsulated from outer scope for compact handling without conflicts with other code.
MVC
A design pattern separated in Models, Views and Controllers for huge applications. They are not dependent on each other and can be changed/improved/replaced without to take care of other code.
Example:
Car (Model) The object that is being
presented.
Example in IT: A HTML form.
Camera (View) Something that is able to see the object(car).
Example in IT: Browser that renders a website with the form.
Driver (Controller) Someone who drives that car.
Example in IT: Functions which handle form data that's being submitted.
Snippets
Small codes of only a few lines, may not be even complete but worth for a quick share.
Plug-ins
Exclusive functions for specified frameworks/APIs/libraries only.
Add-ons
Additional modules or services for specific GUIs.
SDK represents to software development kit, and IDE represents to integrated development environment. The IDE is the software or the program is used to write, compile, run, and debug such as Xcode. The SDK is the underlying engine of the IDE, includes all the platform's libraries an app needs to access. It's more basic than an IDE because it doesn't usually have graphical tools.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I recently discovered Erlang and am now working my way through a couple of tutorials. By now I'm looking forward to actually implement something as a hobby project. I'm not really interested in yet another chat server. I would like to code something more interesting (yes I'm aware that this is a rather fuzzy term) which is also manageable, so I can finish it in my spare time.
Any suggestions?
Edit: The project should preferably highlight Erlang's strenghts (concurrency, distributed).
Build a distributed system that searches twitter feeds in real time and allows anyone to perform searches from a web front end.
Build a distributed file system. Implement distributed B*Trees or B+Trees as the base of this file system. Do it in erlang.
Build a distributed key value store on top of the distributed file system built in step 2.
Build a distributed web index (to be used by a distributed web search engine) on top of the key value store.
Build a distributed linker. Advanced build automation offers remote agent processing for distributed builds and/or distributed processing.
Build a MMORPG backend that relies on distributed storage of the game/player state and distributed processing of user requests.
For something for yourself, consider writing a simple server; something that, for example, services date/time requests or -- a little fancier -- an HTTP daemon that serves only static content.
The best part of Erlang is the way it handles concurrency; exercize that.
Project Euler, for sure.
Some things from my copious ToDo list that would both be good learning exercises and helpful to the erlang community at large:
Profile all the available Key/Value stores:
Write a library for testing insert, lookup, delete, search times for a variety of K/V stores
Create a benchmark suite people can run
Make it work with ets, dets, proplists, gb_trees, dict, orddict, redblack trees, bdb, tokyocabinet, ...
Produce pretty graphs
Make it easy to update, contribute to and run on anyone's machine
write a new io_lib:format routine that uses named parameters:
io_lib:nformat("Hi there ~{name}s~n.", [{name, "Bob"}]).
This is useful for internationalisation if the position of parameters changes when the language of the format string changes.
Extend erl -make (make.erl)
Allow adding code paths (so that you don't need to do erl -pa LibraryPath -make)
Compile/load behaviour modules before modules that implement those behaviours
Handle hierarchal modules correctly (output path in particular)
This doesn't exactly answer your question, but if you are looking for an interesting free, open-source project that is written in Erlang, you should definitely check out CouchDB. From the website:
Apache CouchDB is a distributed,
fault-tolerant and schema-free
document-oriented database accessible
via a RESTful HTTP/JSON API. Among
other features, it provides robust,
incremental replication with
bi-directional conflict detection and
resolution, and is queryable and
indexable using a table-oriented view
engine with JavaScript acting as the
default view definition language.
CouchDB is written in Erlang, but can
be easily accessed from any
environment that provides means to
make HTTP requests. There are a
multitude of third-party client
libraries that make this even easier
for a variety of programming languages
and environments.
The CouchDB website has more details. Happy coding!
find something erlang doesn't have that you understand and like. I did that with etap https://github.com/ngerakines/etap/ Now nick has taken over management and it's used internally at EA games. It was fun to make and like a previous poster it was something real so I learned to serve real world problems working on it.
File indexing/search system. This was going to by intro project but I've switched over to something else.
Once you've got it working you could move the indexes to mnesia, and then spread the thing out other nodes to a have a whole network index.