Add job script validations

[qianyuanzhu]

Overview

• Add job script validation and provide recommendation for users

• Step1: Omit files larger than 65KB for this field

• Step2: Group files to "suggested" and "others" and sort alphabetically

• Suggested files meet any of the following criteria
  1. File name extension is one of [".sh", ".job", ".slurm", ".batch", ".qsub", ".sbatch", ".srun", ".bsub"]
  2. First line of the file is a shebang line
  3. First 1000 bytes of the file contain '#PBS' or '#SBATCH"

• Others files are files not omitted in Step1 and not meet requirement to be suggested

Tests

• Tested the example on the issue page
• Tested the job script under the sub folder
  

Fixes #105

[ericfranz]

could cache this like:

def folder_contents
  return @folder_contents if defined? @folder_contents

  ...
end

[ericfranz]

This actually only works if you set the @folder_contents otherwise. For example:

def folder_contents
  return @folder_contents if defined? @folder_contents

  if File.directory?(self.staged_dir)
    @folder_contents = Find.find(self.staged_dir).drop(1).select {|f| File.file?(f) }.map {|f| WorkflowFile.new(f, self.staged_dir)}
  else
    @folder_contents = []
  end
end

[qianyuanzhu]

Thank you for your feedback! I cleaned up the code and added error handling for the file permission.

Another issue I had is that the titles for option groups will still be displayed when no options available under that group. It's handled by removing empty option groups.

1. Have both suggested and other files:
   

2. Only have suggested files:
   

3. Have valid but not suggested files:
   I changed the group title "Others" to "Other valid file(s)" because "Others" could be confusing when it's the only group available. We can also remove the "Other valid file(s)" title completely if that's more user-friendly.
   

4. No valid files or folder is empty:
   

[qianyuanzhu]

Thank you for the feedback and help identify the script directives! I updated v.blank? to v.empty?, added the checks for the new resource manager directives, and opened the issue you mentioned: OSC/ood_core#161.

[MorganRodgers]

For the contents.include? predicate I would prefer:

['#PBS', '#SBATCH', '#BSUB', '#$'].any?{ |directive| contents.include?(directive) }

[MorganRodgers]

Although this is fine as is: a better design here would be for ood_core's Adapters to provide their directive as a constant and for this array to be built from the available clusters. The more tests we have the more likely we are to get a false positive.

[MorganRodgers]

Just noticed OSC/ood_core#161. +1

[MorganRodgers]

Pointless whitespace...

[MorganRodgers]

This works for me.

• Ideally we would build the list of resource manager directives from OODClusters after ood_core has been updated.
• While this is a good start the workflow to get to here is not what I would expect as a user. I would expect that when creating a new workflow from an existing directory I would be able to select my script there. We could add this to the new_from_path view by adding some JS to listen to the change event on workflow[staging_template_dir], and then updating the script select options via AJAX.

Those changes can be a separate PR.