The US Army Corps of Engineers Research and Development Center (ERDC) is conducting market research for the on-ramping of additional firms to the High Performance Computing Modernization Program (HPCMP) Basic Ordering Agreeement (BOA) in support of Technical Insertion (TI) requirements at its DoD Supercomputing Resource Centers (DSRCs).

ERDC is requesting interested firms to submit a capabilities package in response to this announcement for review of possible market capabilities to perform its TI requirements. The description of the work is provided below which also will include the delivery, assembly, acceptance testing of a fully operational system, system administration (if required in the overall TI requirements) and hardware/software maintenance as specified in the individual TI requirements.

NOTE: System administration of the systems provided may or may not be required for all future requirements. System administration requirements will be identified in the specific order requirements as they materialize. Those firms that can provide the equipment/systems and provide hardware/software maintenance only, are encouraged to provide their capabilities statement to execute such requirements.

****For the systems to be procured for FY24, order requirements are projected to be the systems and hardware/software maintenance only of those fully operational systems. No system administration will be requested.

TECHNOLOGY INSERTION (TI) DESCRIPTION OF WORK:

1.0 OBJECTIVE

The primary objective of this Basic Ordering Agreement (BOA) is to provide the High Performance Computing Modernization Program (HPCMP) with world-class high performance computing capabilities for the United States Department of Defense (DOD). The BOA holders will be responsible for providing the Government with balanced, commercially- available, production-grade High Performance Computing (HPC) systems, which contain an appropriate combination of processor, memory, disk Input/output (I/O), interconnect, and Operating System (OS) capabilities in order to conduct complex, tightly coupled, large-scale, scientific calculations.

2.0 DESCRIPTION

2.1 Locations

Requirements for each order will be located primarily at the following four DSRCs and will encompass the unique requirements of each center:

• • Air Force Research Laboratory (AFRL) DSRC, Wright-Patterson Air Force Base, Ohio
  • Army Research Laboratory (ARL) DSRC, Aberdeen Proving Ground, Maryland
  • U.S. Navy DSRC, John C. Stennis Space Center, Mississippi
  • U.S. Army Corps of Engineers, Engineer Research and Development Center (ERDC) DSRC, Vicksburg, Mississippi

2.2 System Manufacturing Capabilities

To satisfactorily complete the work, the contractor, who must be a United States of America Original Equipment Manufacturer (OEM), must have expert knowledge of Federal Government standards and industry recognized national standards for advanced computing and communication technologies. Targeted system attributes at the order level may include the following with specific orders may have varying requirements. These are meant to be illustrative of an exemplar HPC system.

• • A high ratio of peak CPU memory bandwidth to peak CPU floating point capability.
  • 256 GB to 4 TB of random access memory per compute node. Single bit correction and multibit error detection are required.
  • A minimum of 512 compute total compute nodes spanning a range of node types to include a range of supported processor socket counts and nodes with and without support for PCIe devices (e.g. non volatile memory express (NVMe) storage, general purpose graphic processing units (GPGPUs))
  • Multiple petabyte-scale distinct parallel file systems with all metadata stored on solid state devices, no single point of failure, and protection from double storage device failures (e.g RAID6)
  • Aggregate data transfer rates between the compute nodes and each file systems ranging from 20 GB/s to 1 TB/s
  • An ability to simultaneously mount multiple file systems on all login nodes and all compute nodes or a subset of file systems on a subset of nodes.
  • A high number (500,000+) of I/O operations per second (IOPs) using 1, 32, and 128 active nodes for the following file operations for each parallel file system: (1) open/create, (2) file stat, (3) #1 and #2 simultaneously, and (4) unlink.
  • A tightly integrated interconnect with (1) each link having at least 100 Gbps of bandwidth per processor socket and (2) an end-to-end latency that is no greater than 3µs (two microseconds)
  • A well-tuned operating system (OS) with low jitter, configured in accordance with DISA security technical implementation guides (STIG).
  • A 10/40/100 gigabit Ethernet interface (or higher data rate) for external (i.e. wide-area network [WAN]) connectivity with IPv4 and IPv6 dual-stack functionality
  • A 10/40/100 gigabit Ethernet interface (or higher data rate) for internal DSRC (i.e. local-area network [LAN]) connectivity with IPv4 and IPv6 dual-stack functionality
  • Workload management software with ability to launch jobs using container-based solutions (e.g. Docker, Shifter, Singularity)
  • An appropriate number of job scheduling nodes with (a) suitable job scheduling software and (b) adequate memory, processing capability, and redundancy
  • Use of HPCMP Kerberos and HPCMP Public Key Infrastructure (PKI) authentication software and HPCMP Secure Shell (SSH) communication software which will be furnished by the Government.
  • Adherence to the following power quality standards: CBEMA, ITIC, SEMI F47, IEC 61000-4-11/34
  • Three-phase power cabinet supply with a preference for a supply voltage of 480V
  • A cooling solution in which a minimum of 98% of the heat generated by the system is removed by liquid- cooling.
  • Login nodes and data transfer nodes which contain the same processor type as the compute nodes to facilitate compiling, data transfer, and other interactive functions, but contain more memory per core, additional 10/40/100 Gbps network interfaces, and with support for machine learning accelerators (e.g. GPGPUs).
  • A test and development system (TDS) which is a smaller version of the base system, configured with similar hardware, software, file system, and I/O attributes to allow effective system software testing, user code porting, and base system configuration management.
  • An ability to execute as many of the software packages listed on the HPCMP’s software web page as possible: https://centers.hpc.mil/software
  • At least two suites of high-level language compilers: (a) GNU Compiler Collection (version 11 or higher) and (b) a proprietary suite targeting each processor type included in the system
  • At least two implementations of MPI with at least one being compliant with the MPI- 3.1+ standard.
  • JavaScript, Go, and Perl scripting languages. Latest version of Python 3 must be provided with as many of the following extensions as possible: NumPy, MPI4Py, SciPy, PyTorch, and IPython. Support for Jupyter notebooks is required.
  • Multiprocessing APIs (including OpenMP), standard computational libraries (including BLAS, BLACS, FFTW, LAPACK, and ScaLAPACK), accelerator computational libraries/compilers (including CUDA, OpenCL, and OpenACC), performance libraries including the Performance API (PAPI), and an integrated development environment.
  • Application performance instrumentation tools (including at least one code profiler that interfaces with PAPI and at least one tool that provides MPI messaging statistics)
  • A high level of system reliability with a high overall effectiveness level and a low number of user/job interrupts.
  • A high level of performance on high performance applications and libraries such as GAMESS, CTH, HYCOM, and FFTW over processor ranges from 8,192 to a minimum of 65,536 processor cores

2.3 Administration and Maintenance

System Administration of the systems may be required as part of the overall specific system order requirements. When required, System Administration shall be performed by the Offeror 24 hours per day, 7 days per week (over five one-year options at the individual order level) with BOA holders ensuring that (1) the monthly system effectiveness level does not fall below 97%, and (2) the monthly number of user/job interrupts does not exceed the maximum allowable (which is determined by dividing the number of nodes in the base system by one hundred). Requirements for System Administration will be dependent on the specific order and shall be identified for the order/system in any Request for Proposal submitted to the BOA holders to support individual Technical Insertion requirements.

In all requirements, Hardware and Software maintenance for the systems shall be maintained by the Offeror with 24 hours per day, 7 days per week system maintenance (over five one-year options at the individual order level) and therefore must ensure that (1) the monthly system effectiveness level does not fall below 97%, (2) the response time for remedial maintenance is less than four hours (some orders may require a short response time), (3) new software is tested and ready for operation prior to implementation on the base system, and (4) software/hardware security vulnerabilities are addressed in a timely manner.

3.0 SECURITY REQUIREMENTS

Since HPCMP systems will be placed (predominantly) at DOD facilities, BOA holders must have sufficient personnel with SECRET level clearances for the purposes of system installation, capability testing, effectiveness level testing, administration, and maintenance. All personnel requiring privileged access to any system and/or supporting network/security component must have an adjudicated tier 5 background investigation, a clearance commensurate with the classification of the system, Information Assurance Technician (IAT) Level II or III certification (see https://disa.mil/ or https://disa.mil/NewsandEvents/Training) and computing environment certifications specific to all relevant operating systems.

4.0 PUBLIC AFFAIRS

The BOA holder shall not publicly disclose any data generated or reviewed under this agreement. The BOA holder shall refer all requests for information concerning projects to the Contracting Officer for comment. Prior to release, any publication articles shall be coordinated and approved by the Contracting Officer.

5.0 SUBMITTALS

All submittal requirements will be identified at the order level. These may include but are not limited to: safety plans, quality control plans, project schedules, testing plans, testing results, and closeout documentation.

6.0 REPORTING REQUIREMENTS

All reporting requirements will be established at the order level. Typical reporting requirements are on a monthly basis however, each order will identify the specific requirements for that procurement.

7.0 POINTS OF CONTACT

The POCs for the BOA level are listed below. Other POCs may be identified at the individual order level. The Contracting Officer shall always be included on all correspondence related to this BOA and all orders.

7.1 Contracting Officer:

7.2 Acquisition Project Manager:

The applicable NAICS code for this procurement is 334111 and is provided as part of this announcement. This is a Sources Sought open to all qualified prime contractor firms (Large and Small Business under NAICS 334111). All interested firms are encouraged to respond to this announcement with submission by 06 September 2023, 1700 EST to the Points of Contact listed in this announcement.

Interested firms should submit a capabilities package (not exceeding 5 pages) demonstrating the ability to perform and meet the requirements listed above. Packages should include the following information:

(1) Business name, address, point of contact including email addresss, and business size under the NAICS 334111;

(2) Identification of business type (i.e., Large Business, Small Business, 8(a), HUBZone, SDVOSB, Woman Owned Small Business, etc.);

(3) CAGE Code;

(4) Demonstration of the firms experience as a PRIME contractor on projects of similar type and complexity of this requirement within the past five (5) years. Please list actual projects completed of similar type equipment and follow on services as described in the description of work. For those projects completed, include project title and location, a brief description of the project to include the overall dollar value and whether the work was self-performed.

Responsible sources demonstrating relevant experience and capabilities to perform the identified requirements will be considered in our overall market research. System for Award Management (SAM), as required by FAR 4.1102 and FAR 4.1201 shall apply to this requirement. Prospective contractors must be registered for consideration. Lack of registration in SAM will render a firm ineligible for award of any potential BOA.

Attachments/Links