Link to dynamically loaded assemblies in dynamically compiled code

I have a situation here in a .net project in which I compile C # scripts in assemblies in memory using CodeDomProvider.

In the CompilerParameters class, which I use to compile, I refer to some assemblies by name (System.dll and one that is part of the project), and everything still works. However, some scripts use some code (through the interface and polymorphism), which is contained in the assembly, which is precompiled, but dynamically loaded from disk (for example, a plug-in). In this situation, this will not work, and the problem is that I do not know how to reference the dynamically loadable assembly when the script is dynamically compiled.

If I put my script in the class library and from this class library, I reference the plugin assembly, everything works fine, but if I add the plugin assembly name (dll file) to the reference assemblies in CompilerParameters and compile the script dynamically after that, the script is executed, but the code that is in the plugin dll is missing.

I hope it is clear what I am trying to do here, please help if you know anything about this, because I have been afraid for a while, and nothing that I tried works.

Thank!

Edit: Here is the code to download the dll plugins:

        DirectoryInfo di = new DirectoryInfo(@".\Plugins");
        FileInfo[] files = di.GetFiles("*.dll");
        foreach (FileInfo fi in files)
        {
            try
            {
                //load all dll files from the app pack directory
                Assembly asm = Assembly.LoadFrom(fi.FullName);
                Assemblies.Add(asm);
                foreach (Type type in asm.GetTypes())
                {
                    try
                    {
                        object instance = null;
                        instance = Activator.CreateInstance(type);

After that, I save the instance in the dictionary for future use. On the other hand, here is how I try to pass script links that will be compiled:

        CodeDomProvider provider = CodeDomProvider.CreateProvider("CSharp");
        List<string> referenceAssemblies = new List<string>();
        referenceAssemblies.Add("System.dll");
        referenceAssemblies.Add("VFS.dll");
        foreach (Assembly asm in PluginManager.Instance.Assemblies)
        {
            referenceAssemblies.Add(asm.Location);
        }
        CompilerParameters compilerParameters = new CompilerParameters(referenceAssemblies.ToArray());

This works if I do not use any class that is part of the plugin.