Empty metadata in results of circuit simulation - cuquantum appliance

[intelligi123]

Hi,

I am interested in finding out how much memory has been consumed by the circuit while single and multiple node simulation, however when I printed result variable, it doesn't show anything in metadata to find about exact memory utilization. How can i find that?

Also sometimes running circuits, shows almost all of GPU memory usage but GPU-Util is only upto half and vice versa (less Memory-Usage and almost all of GPU-Util) in result of nivdia-smi. What is the difference in terms of circuit execution in these scenarios?

I am using two nodes, each has two A40 GPUs.

Besides this I have some other queries regarding memory usage:

1. Can I make use of system memory while working with cuquantum appliance or only GPU memory is available?
2. Can I execute a circuit on multiple nodes with different GPUs installed?

Thank you

[yangcal]

Hello,

Can you provide more details on the routines that you're using for your simulation? Eg, what simulation methods you're using and what metadata you're looking for etc. Would be best if you could share a minimal script/command lines showing what you're aiming for.

[intelligi123]

Sure,

I am running this script which is simply a ghz state creation on two nodes

from qiskit import QuantumCircuit, transpile
from qiskit import Aer
from mpi4py import MPI
import argparse

def create_ghz_circuit(n_qubits):
    circuit = QuantumCircuit(n_qubits)
    circuit.h(0)
    for qubit in range(n_qubits - 1):
        circuit.cx(qubit, qubit + 1)
    return circuit

def run(n_qubits, precision, use_cusvaer):
    simulator = Aer.get_backend('aer_simulator_statevector')
    simulator.set_option('cusvaer_enable', use_cusvaer)
    simulator.set_option('precision', precision)
    circuit = create_ghz_circuit(n_qubits)
    circuit.measure_all()
    circuit = transpile(circuit, simulator)
    job = simulator.run(circuit)
    result = job.result()

    if MPI.COMM_WORLD.Get_rank() == 0:
        print(f'precision: {precision}')
        print(result)
        print(f'backend: {result.backend_name}')

parser = argparse.ArgumentParser(description="Qiskit ghz.")
parser.add_argument('--nbits', type=int, default=20, help='the number of qubits')
parser.add_argument('--precision', type=str, default='single', choices=['single', 'double'], help='numerical precision')
parser.add_argument('--disable-cusvaer', default=False, action='store_true', help='disable cusvaer')

args = parser.parse_args()

run(args.nbits, args.precision, not args.disable_cusvaer)

Here is the response I am getting:

precision: single
Result(backend_name='cusvaer_simulator_statevector', backend_version='23.10.0', qobj_id='c28879cd-a68c-4f7e-94fa-3fbd8ec7fea6', job_id='d67934a4-3f34-460e-aeeb-4e71b63f81e9', success=True, results=[ExperimentResult(shots=1024, success=True, meas_level=2, data=ExperimentResultData(counts={'0xfffff': 484, '0x0': 540}), header=QobjExperimentHeader(qubit_labels=[['q', 0], ['q', 1], ['q', 2], ['q', 3], ['q', 4], ['q', 5], ['q', 6], ['q', 7], ['q', 8], ['q', 9], ['q', 10], ['q', 11], ['q', 12], ['q', 13], ['q', 14], ['q', 15], ['q', 16], ['q', 17], ['q', 18], ['q', 19]], n_qubits=20, qreg_sizes=[['q', 20]], clbit_labels=[['meas', 0], ['meas', 1], ['meas', 2], ['meas', 3], ['meas', 4], ['meas', 5], ['meas', 6], ['meas', 7], ['meas', 8], ['meas', 9], ['meas', 10], ['meas', 11], ['meas', 12], ['meas', 13], ['meas', 14], ['meas', 15], ['meas', 16], ['meas', 17], ['meas', 18], ['meas', 19]], memory_slots=20, creg_sizes=[['meas', 20]], name='circuit-122', global_phase=0.0, metadata={}), status=DONE, name='circuit-122', seed_simulator=1386540608, time_taken=1.120253086090088, metadata={})], date=2024-05-08T06:05:42.739882, status=COMPLETED, header=QobjHeader(backend_name='cusvaer_simulator_statevector', backend_version='23.10.0'), mpi_rank=0, num_mpi_processes=16, time_taken=1.1231706142425537, cusvaer_enabled=True)
backend: cusvaer_simulator_statevector

This response has no fields in metadata portion, which normally has this kind of information when I run circuit on out of cuquantum environment

Result(backend_name='aer_simulator', backend_version='0.14.0', qobj_id='', job_id='a557d971-e137-4b59-b23a-ffdc962bd2b7', success=True, results=[ExperimentResult(shots=1024, success=True, meas_level=2, data=ExperimentResultData(counts={'0x0': 530, '0x1fffffff': 494}), header=QobjExperimentHeader(creg_sizes=[['meas', 29]], global_phase=0.0, memory_slots=29, n_qubits=29, name='circuit-164', qreg_sizes=[['q', 29]], metadata={}), status=DONE, seed_simulator=1000, metadata={'time_taken': 137.727448118, 'num_bind_params': 1, 'parallel_state_update': 8, 'parallel_shots': 1, 'required_memory_mb': 8192, 'input_qubit_map': [[28, 28], [27, 27], [26, 26], [25, 25], [24, 24], [23, 23], [22, 22], [21, 21], [20, 20], [19, 19], [18, 18], [17, 17], [16, 16], [15, 15], [14, 14], [13, 13], [0, 0], [1, 1], [2, 2], [3, 3], [4, 4], [5, 5], [6, 6], [7, 7], [8, 8], [9, 9], [10, 10], [11, 11], [12, 12]], 'method': 'statevector', 'device': 'CPU', 'num_qubits': 29, 'sample_measure_time': 1.047521495, 'active_input_qubits': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], 'num_clbits': 29, 'remapped_qubits': False, 'runtime_parameter_bind': False, 'max_memory_mb': 15903, 'noise': 'ideal', 'measure_sampling': True, 'batched_shots_optimization': False, 'fusion': {'applied': True, 'time_taken': 0.015519162, 'cost_factor': 1.8, 'parallelization': 1, 'max_fused_qubits': 5, 'method': 'unitary', 'threshold': 14, 'enabled': True}, 'cacheblocking': {'max_multiple_chunk_swaps': 10, 'multiple_chunk_swaps_buffer_qubits': 15, 'multiple_chunk_swaps_enable': True, 'block_bits': 25, 'enabled': True}}, time_taken=137.727448118)], date=2024-05-08T11:15:34.030914, status=COMPLETED, header=None, metadata={'time_taken_execute': 138.022402091, 'mpi_rank': 0, 'time_taken_parameter_binding': 0.000346595, 'num_mpi_processes': 1, 'num_processes_per_experiments': 1, 'omp_enabled': True, 'max_gpu_memory_mb': 0, 'max_memory_mb': 15903, 'parallel_experiments': 1}, time_taken=138.05865716934204)

Here I get fields like required_memory_mb and max_memory_mb

How can I get this from simulation run on cuquantum appliance? Also I would really appreciate if get answer of other queries mentioned in question (1 and 2).

[ymagchi]

Hi @intelligi123,
Thank you for your questions, I have answered them below.

I am interested in finding out how much memory has been consumed by the circuit while single and multiple node simulation, however when I printed result variable, it doesn't show anything in metadata to find about exact memory utilization. How can i find that?

Currently we return empty metadata in our cusvaer_simulator_statevector backend, and we don't include required_memory_mb item in it. The memory usage can be estimated by the targeting precision and the number of processes and qubits: 8/16 Bytes * (2 ** number_of_qubits) / (number_of_processes) will be used for each distributed state vector.

Also sometimes running circuits, shows almost all of GPU memory usage but GPU-Util is only upto half and vice versa (less Memory-Usage and almost all of GPU-Util) in result of nivdia-smi. What is the difference in terms of circuit execution in these scenarios?

It depends on the sampled timing and could vary during each circuit execution. GPU Utilization is a sampled measurement over a time period, and it just reports what percentage of time one or more GPU kernel(s) was active.

Can I make use of system memory while working with cuquantum appliance or only GPU memory is available?

Current cuQuantum Appliance supports only GPU memory for state vectors.

Can I execute a circuit on multiple nodes with different GPUs installed?

It is possible, however, the workload in the distributed state vector simulation should be equal among processes and the load balance would get worse if different GPUs (architectures) are assigned to the same problem (Please let me know if this is not what you meant in the original question.)

[intelligi123]

Thank you for clarification.