Evaluator

Background

All versions of CS-Script prior v3.5 offered script engine built around .NET CodeDOM. CodeDOM has reasonably "comfortable" and intuitive API and it is fully suitable for the the standalone script execution. From another hand CodeDOM is not as suitable for the script hosting scenarios. The problem is that CodeDOM is a fake "compiler as service". While API makes an impression that the compiler (e.g. CSharpCodeProvider) compiles the source code in memory it is in fact saving the code into temporary file and feeding it to the conventional external compiler process (e.g. csc.exe). Thus the script compilation is a relatively slow process.

Another problem is related to the execution stage. There is this notorious CLR design flaw, which does not allow unloading any assembly once it is loaded. It is a problem because the script when it is compiled is an assembly and it is loaded in the current AppDomain thus if one is not careful the Memory Leak condition can arise.

CS-Script offer a "work around" for the both problems. Compiling can be in many cases completely eliminated by CS-Script caching. And for the unloading assembly CS-Script offers AsmHelper class, which allows execution of the script in the temporary AdppDomain and unloading this temp AdppDomain with the loaded complied script automatically after the script execution.

However the caching is the most advantages for the standalone execution (not hosting). And the script execution in a separate AdppDomain requires all types being passed in and out to be serializable. It attracts some performance penalty associated with the crossing the AppDomain boundaries and more importantly in many cases it imposes the very strong and unnecessary design constraint (serialization). The constraint, which sometimes even impossible to satisfy to (e.g. non-serializable third-party sealed classes cannot be made serializable).

All these problems could be solved only by a proper "compiler as service" solution. However .NET does not have one out of box. The Roslyng project is a good candidate to base the hosting model on. However certain technical decisions Roslyn team made ragarding the product functionality made it impossible to use Roslyng for the C# scripting. Read in details here.

From another hand Mono had "compiler as service" for some time. Its API is not very conventional and it also has some limitations (see further in the text) but it works. Thus in CS-Script v3.5 new script hosting model based on the Mono "compiler as service" (Mono.CSharp.dll) was introduced.

Hosting CS-Script Evaluator

When hosting the CS-Script engine in the applications developers have two options: to use .NET CodeDOM compiler or to use Evaluator based on Mono "compiler as service". Evaluator is a preferred choice as it offers a better compiling performance and superior memory management. Evaluator is in fact the only solution, which finally solves the CLR Memory Leak problem allowing same AppDomain script execution without any class design constraints.

Mono "compiler as service" (Mono.CScript.dll). Always loads compiled C# code (script) into the caller AppDomain thus it is not immune to the CLR Memory Leak problem at all. The same applys to the Loaded/Referenced third-party assemblies. In fact Mono completely ignores the problem and does not even offer any work around as CS-Script does. However there is a vary specific scenario, which allows Mono executing the full scale C# code without constantly growing memory footprint and number of loaded assemblies.

Thus if the various C# code snippets define the same set of types (even with the different implementations) are compiled with the Mono.CScript.Evaluator.Compile() the compiler emits only a single dynamic assembly regardless of how many times Compile is called. Unfortunately all other hosting scenarios lead to the Memory Leaks (see Samples\Hosting\CompilerAsService\MemoryManagement.cs sample).

The Mono "compiler as service" API is tightly coupled to the other Mono modules and highly specialized for the use in Mono tools. Also it is not very convenient for the real live hosting scenarios (e.g. compiling of the invalid C# code never return any error nor throw the exception but creates the log entry instead). Thus CS-Script implements the API adapter (CSScript.Evaluator), which not only makes the API more consistent with the CS-Script hosting API model but also automates the many routine script setup tasks (e.g. assembly auto-referencing, error handling).

It is very easy to distinguish in the written hosting code if CodeDOM or Evaluator is used for the script execution. All CSScript.Evaluator.* compile/load calls are passing the scripts to the "compiler as service" and all CSScript.* ones to the CodeDOM compiler.

The following are the fragments from the <cs-script>\Samples\Hosting\CompilerAsService\HostingScenarios.cs sample, which demonstrates the all possible hosting scenarios in details.

Loading a script class:

dynamic script = CSScript.Evaluator
                         .LoadCode(@"using System;
                                     public class Script
                                     {
                                         public int Sum(int a, int b)
                                         {
                                             return a+b;
                                         }
                                     }");
int result = script.Sum(1, 2);

Loading a script class and aligning it to the interface. Note the script class doesn't have to inherit from the interface.
Though if the script class indeed inherits from the interface then a simple type casting is preferred ( var calc = (ICalc)CSScript.Evaluator.LoadCode(...)).

ICalc calc = CSScript.Evaluator
                     .LoadCode<ICalc>(@"using System;
                                      public class Script
                                      {
                                          public int Sum(int a, int b)
                                          {
                                              return a+b;
                                          }
                                      }");
int result = calc.Sum(1, 2);

Loading a classless script:

dynamic script = CSScript.Evaluator
                         .LoadMethod(@"int Product(int a, int b)
                                       {
                                        return a * b;
                                       }");
int result = script.Product(1, 2);

Passing the host type reference to the script:

public class HostApp
{
    public string Name; 
 
    public void Test()
    { 
        dynamic script = CSScript.Evaluator
                                 .LoadMethod(@"void Print(HostApp host)
                                              {
                                                  Console.WriteLine(host.Name);
                                              }");
       this.Name = "ScriptHost";
       script.Print(this);
   }
}

Evaluating the C# statement. This is arguably the most canonical Eval sample, at the same time it is the list practical one. The reason is that it is not possible to directly pass any arguments to the script (statement).

int result = (int)CSScript.Evaluator.Evaluate("1 + 2;");

Limitations

When using CSScript.Evaluator some special considerations should be taken into account. In order to avoid Memory Leaks a minor code design constraint should be satisfied. Thus when invoking LoadCode/LoadFile you should try to reuse the same top level class name to avoid emitting unnecessary assemblies. This technique is enforced when LoadMethod is invoked. See <cs-script>\Samples\Hosting\CompilerAsService\MemoryManagement.cs for the details on haw script code structure affect the memory consumption.

There is also another limitation (rather a quality) of the hosting with "compiler as service". Debugging of the script code is impossible. This is because of the very nature of the "compiler as service" engine, which does not save the script code to the file and in result there is no possibility for the debugger to map the IL instruction to the C# statement.

Interestingly enough there are a few posts on Mono forums asking if there any way to overcome this Mono limitation. Sadly these posts have no single response:
http://www.thebirdietoldme.com/userActions/thread/Question.aspx?id=11816467
http://stackoverflow.com/questions/11816467/debugging-code-compiled-using-the-mono-csharp-evaluator
...

Note that this does not mean that it is not possible to compile the script as DEBUG. It is in fact quite simple:

CSScript.Evaluator.Configuration = BuildConfiguration.Debug;

However this will only enforce DEBUG and TRACE conditional symbols but will not yield a proper PDB file. The technique is demonstrated in Evaluator_Debug method of the <cs-script>\Samples\Hosting\CompilerAsService\HostingScenarios.cs sample. Thus if the full scale debugging is vital then it is recommended to do the temporary switch to CodeDOM compiler to do the debugging as it is demonstrated in the CodeDOM_Debug() method of the same sample file.

The following table contains brief comparison of the script hosting features of CodeDOM and Evaluator hosting model. The table is largely composed on the runtime data collected whith <cs-script>\Samples\Hosting\CompilerAsService\MemoryManagement.cs. While Evaluator ("compiler as service") is the recommended approach the table below may help you to make your own decision on the choice of the hosting model.

The data was collected for Mono.CSharp.dll of Mono v3.2.3

Fearure	CodeDOM	Evaluator
Compiling performance	slower	faster
Runtime performance	fast	fast
Avoiding Memory Leaks in the same AppDomain scenarios for the fixed class name script	impossible	possible*
Avoiding Memory Leaks in the same AppDomain scenarios for the variable class name script	impossible	impossible
"No Memory Leaks" mode constraints	remote AppDomain is required	fixed class name
Debugging	possible	impossible

*There were a couple of the reports describing Mono.CScript.Evaluator.Compile() still leaking when used for scaled up scenarios from CompilerAsService\MemoryManagement.cs.

       
It is highly recommended that you carefully investigate the following code samples:

<cs-script>\Samples\Hosting\CompilerAsService\MemoryManagement.cs
Demonstrates runtime memory and performance implications depending on the chosen hosting API.
<cs-script>\Samples\Hosting\CompilerAsService\HostingScenarios.cs
Demonstrates usage of CSScript.Evaluator in all possible (supported) hosting scenarios.

Be also aware that because of the true "in-memory" nature of CSScript.Evaluator assembly hosting there is no any sort of caching simmilar to CS-Script caching for CodeDOM.

Evaluator - Compiler As Service

Background

Hosting CS-Script Evaluator