Auctioning Cluster Resources

Lee Or Alon; Orna Agmon Ben-Yehuda; Sigal Oren

doi:10.1145/3502181.3533713

Auctioning Cluster Resources

Lee Or Alon, Orna Agmon Ben-Yehuda, Sigal Oren

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Organizational clusters are often shared among users who compete over resources, but the organization's goal is to increase the overall utility produced by the cluster. That is, to increase the aggregate benefit drawn from the cluster. To overcome this problem, we enhanced SLURM with an auctioning system. We evaluated our work on several real cluster traces. Our auctioning scheduler increases the responsiveness to highly valued jobs. It reduces the weight of the queued jobs (the sum of multiplication of jobs by their wait time by their number of required nodes and by their bid) by 3X - 33X compared with backfilling.

Original language	English
Title of host publication	HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing
Publisher	Association for Computing Machinery, Inc
Pages	285-286
Number of pages	2
ISBN (Electronic)	9781450391993
DOIs	https://doi.org/10.1145/3502181.3533713
State	Published - 27 Jun 2022
Externally published	Yes
Event	31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2022 - Virtual, Online, United States Duration: 27 Jun 2022 → 30 Jun 2022

Publication series

Name	HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing

Conference

Conference	31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2022
Country/Territory	United States
City	Virtual, Online
Period	27/06/22 → 30/06/22

Bibliographical note

Publisher Copyright:
© 2022 Owner/Author.

Funding

This research was partially supported by ISF grant 2167/19. The authors would like to thank M. Ben-Yehuda, S. Dobzinski, N. Arbel, G. Oren, M. Kanovsky, Y. Perry and the users who kindly participated in our interviews and survey.

Funders	Funder number
Israel Science Foundation	2167/19

Keywords

auction
cluster resources
slurm

Access to Document

10.1145/3502181.3533713

Cite this

Alon, L. O., Ben-Yehuda, O. A., & Oren, S. (2022). Auctioning Cluster Resources. In HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing (pp. 285-286). (HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing). Association for Computing Machinery, Inc. https://doi.org/10.1145/3502181.3533713

Alon, Lee Or ; Ben-Yehuda, Orna Agmon ; Oren, Sigal. / Auctioning Cluster Resources. HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. Association for Computing Machinery, Inc, 2022. pp. 285-286 (HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing).

@inproceedings{90b02e5b079640148f717b9902c2c7a3,

title = "Auctioning Cluster Resources",

abstract = "Organizational clusters are often shared among users who compete over resources, but the organization's goal is to increase the overall utility produced by the cluster. That is, to increase the aggregate benefit drawn from the cluster. To overcome this problem, we enhanced SLURM with an auctioning system. We evaluated our work on several real cluster traces. Our auctioning scheduler increases the responsiveness to highly valued jobs. It reduces the weight of the queued jobs (the sum of multiplication of jobs by their wait time by their number of required nodes and by their bid) by 3X - 33X compared with backfilling.",

keywords = "auction, cluster resources, slurm",

author = "Alon, {Lee Or} and Ben-Yehuda, {Orna Agmon} and Sigal Oren",

note = "Publisher Copyright: {\textcopyright} 2022 Owner/Author.; 31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2022 ; Conference date: 27-06-2022 Through 30-06-2022",

year = "2022",

month = jun,

day = "27",

doi = "10.1145/3502181.3533713",

language = "אנגלית",

series = "HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing",

publisher = "Association for Computing Machinery, Inc",

pages = "285--286",

booktitle = "HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing",

}

Alon, LO, Ben-Yehuda, OA & Oren, S 2022, Auctioning Cluster Resources. in HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing, Association for Computing Machinery, Inc, pp. 285-286, 31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2022, Virtual, Online, United States, 27/06/22. https://doi.org/10.1145/3502181.3533713

Auctioning Cluster Resources. / Alon, Lee Or; Ben-Yehuda, Orna Agmon; Oren, Sigal.
HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. Association for Computing Machinery, Inc, 2022. p. 285-286 (HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Auctioning Cluster Resources

AU - Alon, Lee Or

AU - Ben-Yehuda, Orna Agmon

AU - Oren, Sigal

PY - 2022/6/27

Y1 - 2022/6/27

N2 - Organizational clusters are often shared among users who compete over resources, but the organization's goal is to increase the overall utility produced by the cluster. That is, to increase the aggregate benefit drawn from the cluster. To overcome this problem, we enhanced SLURM with an auctioning system. We evaluated our work on several real cluster traces. Our auctioning scheduler increases the responsiveness to highly valued jobs. It reduces the weight of the queued jobs (the sum of multiplication of jobs by their wait time by their number of required nodes and by their bid) by 3X - 33X compared with backfilling.

AB - Organizational clusters are often shared among users who compete over resources, but the organization's goal is to increase the overall utility produced by the cluster. That is, to increase the aggregate benefit drawn from the cluster. To overcome this problem, we enhanced SLURM with an auctioning system. We evaluated our work on several real cluster traces. Our auctioning scheduler increases the responsiveness to highly valued jobs. It reduces the weight of the queued jobs (the sum of multiplication of jobs by their wait time by their number of required nodes and by their bid) by 3X - 33X compared with backfilling.

KW - auction

KW - cluster resources

KW - slurm

UR - http://www.scopus.com/inward/record.url?scp=85134169772&partnerID=8YFLogxK

U2 - 10.1145/3502181.3533713

DO - 10.1145/3502181.3533713

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85134169772

T3 - HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing

SP - 285

EP - 286

BT - HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing

PB - Association for Computing Machinery, Inc

T2 - 31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2022

Y2 - 27 June 2022 through 30 June 2022

ER -

Alon LO, Ben-Yehuda OA, Oren S. Auctioning Cluster Resources. In HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. Association for Computing Machinery, Inc. 2022. p. 285-286. (HPDC 2022 - Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing). doi: 10.1145/3502181.3533713

Auctioning Cluster Resources

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this